Re: [edk2-devel] [PATCH V6] BaseTools:Add the spare space FV image size checker

[Fan, ZhijuX]

Hi LiMing,

I agree with this change and will improve it



Any question, please let me know. Thanks.

Best Regards
Fan Zhiju



> -Original Message-
> From: Gao, Liming 
> Sent: Wednesday, April 15, 2020 9:21 PM
> To: Fan, ZhijuX ; devel@edk2.groups.io
> Cc: Feng, Bob C 
> Subject: RE: [PATCH V6] BaseTools:Add the spare space FV image size checker
> 
> Zhiju:
>   Thanks for your update. The change is good to me. For the report error
> message, I suggest as below
> 
> xx FV free space  is not enough to meet with the required spare space 
> set by -D FV_SPARE_SPACE_THRESHOLD option.
> 
> Thanks
> Liming
> > -Original Message-
> > From: Fan, ZhijuX 
> > Sent: Wednesday, April 15, 2020 7:49 PM
> > To: devel@edk2.groups.io
> > Cc: Gao, Liming ; Feng, Bob C
> > 
> > Subject: [PATCH V6] BaseTools:Add the spare space FV image size
> > checker
> >
> > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654
> >
> > If FV is placed in FD region, its FV image size is fixed.
> > When FV image size exceeds it, it will trig the build break.
> > To alert the developer to adjust FV image size earlier, I request to
> > add new checker for the the spare FV space.
> > When the spare FV space is less than the specified threshold, build
> > tool will report the error.
> >
> > This checker is the optional.
> > It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=1.
> > Macro is the value of the spare space threshold size.
> > It can be decimal or hex format. If it is enabled, BaseTools will
> > check every FV with the fixed size.
> > If FV doesn't meet with the size requirement, Build tool will report
> > error message to say there is no enough spare space.
> >
> > Cc: Liming Gao 
> > Cc: Bob Feng 
> > Signed-off-by: Zhiju.Fan 
> > ---
> > changed the error message
> >
> >  BaseTools/Source/Python/Common/BuildToolError.py |  2 +
> >  BaseTools/Source/Python/build/build.py   | 47 ++
> >  2 files changed, 49 insertions(+)
> >
> > diff --git a/BaseTools/Source/Python/Common/BuildToolError.py
> > b/BaseTools/Source/Python/Common/BuildToolError.py
> > index ecc83d0f48bd..21549683cd19 100644
> > --- a/BaseTools/Source/Python/Common/BuildToolError.py
> > +++ b/BaseTools/Source/Python/Common/BuildToolError.py
> > @@ -64,6 +64,8 @@ COMMAND_FAILURE = 0x7000
> >
> >  PERMISSION_FAILURE = 0x8000
> >
> > +FV_FREESIZE_ERROR = 0x9000
> > +
> >  CODE_ERROR = 0xC0DE
> >
> >  AUTOGEN_ERROR = 0xF000
> > diff --git a/BaseTools/Source/Python/build/build.py
> > b/BaseTools/Source/Python/build/build.py
> > index bec848a7b2e3..68f5b8cabea3 100755
> > --- a/BaseTools/Source/Python/build/build.py
> > +++ b/BaseTools/Source/Python/build/build.py
> > @@ -25,6 +25,7 @@ import traceback
> >  import multiprocessing
> >  from threading import Thread,Event,BoundedSemaphore  import threading
> > +from linecache import getlines
> >  from subprocess import Popen,PIPE, STDOUT  from collections import
> > OrderedDict, defaultdict
> >
> > @@ -1413,6 +1414,9 @@ class Build():
> >  if Target == 'fds':
> >  if GenFdsApi(AutoGenObject.GenFdsCommandDict, self.Db):
> >  EdkLogger.error("build", COMMAND_FAILURE)
> > +Threshold = self.GetFreeSizeThreshold()
> > +if Threshold:
> > +self.CheckFreeSizeThreshold(Threshold,
> > + AutoGenObject.FvDir)
> >  return True
> >
> >  # run
> > @@ -2311,6 +2315,9 @@ class Build():
> >  GenFdsStart = time.time()
> >  if GenFdsApi(Wa.GenFdsCommandDict, self.Db):
> >  EdkLogger.error("build", COMMAND_FAILURE)
> > +Threshold = self.GetFreeSizeThreshold()
> > +if Threshold:
> > +self.CheckFreeSizeThreshold(Threshold,
> > + Wa.FvDir)
> >
> >  #
> >  # Create MAP file for all platform FVs after 
> > GenFds.
> > @@ -2322,6 +2329,46 @@ class Build():
> >  #
> >  self._SaveMapFile(MapBuffer, Wa)
> >  self.CreateGuidedSectionToolsFile(Wa)
> > +
> > +## GetFreeSizeThreshold()
> > +#
> > +#   @retval int Threshold value
> > +#
> > +def GetFreeSizeThreshold(self):
> > +Threshold = None
> &

[edk2-devel] [PATCH V6] BaseTools:Add the spare space FV image size checker

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654

If FV is placed in FD region, its FV image size is fixed.
When FV image size exceeds it, it will trig the build break.
To alert the developer to adjust FV image size earlier,
I request to add new checker for the the spare FV space.
When the spare FV space is less than the specified threshold,
build tool will report the error.

This checker is the optional.
It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=1.
Macro is the value of the spare space threshold size.
It can be decimal or hex format. If it is enabled,
BaseTools will check every FV with the fixed size.
If FV doesn't meet with the size requirement,
Build tool will report error message to say there is no
enough spare space.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
changed the error message

 BaseTools/Source/Python/Common/BuildToolError.py |  2 +
 BaseTools/Source/Python/build/build.py   | 47 ++
 2 files changed, 49 insertions(+)

diff --git a/BaseTools/Source/Python/Common/BuildToolError.py 
b/BaseTools/Source/Python/Common/BuildToolError.py
index ecc83d0f48bd..21549683cd19 100644
--- a/BaseTools/Source/Python/Common/BuildToolError.py
+++ b/BaseTools/Source/Python/Common/BuildToolError.py
@@ -64,6 +64,8 @@ COMMAND_FAILURE = 0x7000

 PERMISSION_FAILURE = 0x8000

+FV_FREESIZE_ERROR = 0x9000
+
 CODE_ERROR = 0xC0DE

 AUTOGEN_ERROR = 0xF000
diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index bec848a7b2e3..68f5b8cabea3 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -25,6 +25,7 @@ import traceback
 import multiprocessing
 from threading import Thread,Event,BoundedSemaphore
 import threading
+from linecache import getlines
 from subprocess import Popen,PIPE, STDOUT
 from collections import OrderedDict, defaultdict

@@ -1413,6 +1414,9 @@ class Build():
 if Target == 'fds':
 if GenFdsApi(AutoGenObject.GenFdsCommandDict, self.Db):
 EdkLogger.error("build", COMMAND_FAILURE)
+Threshold = self.GetFreeSizeThreshold()
+if Threshold:
+self.CheckFreeSizeThreshold(Threshold, AutoGenObject.FvDir)
 return True

 # run
@@ -2311,6 +2315,9 @@ class Build():
 GenFdsStart = time.time()
 if GenFdsApi(Wa.GenFdsCommandDict, self.Db):
 EdkLogger.error("build", COMMAND_FAILURE)
+Threshold = self.GetFreeSizeThreshold()
+if Threshold:
+self.CheckFreeSizeThreshold(Threshold, Wa.FvDir)

 #
 # Create MAP file for all platform FVs after GenFds.
@@ -2322,6 +2329,46 @@ class Build():
 #
 self._SaveMapFile(MapBuffer, Wa)
 self.CreateGuidedSectionToolsFile(Wa)
+
+## GetFreeSizeThreshold()
+#
+#   @retval int Threshold value
+#
+def GetFreeSizeThreshold(self):
+Threshold = None
+Threshold_Str = 
GlobalData.gCommandLineDefines.get('FV_SPARE_SPACE_THRESHOLD')
+if Threshold_Str:
+try:
+if Threshold_Str.lower().startswith('0x'):
+Threshold = int(Threshold_Str, 16)
+else:
+Threshold = int(Threshold_Str)
+except:
+EdkLogger.warn("build", 'incorrect value for 
FV_SPARE_SPACE_THRESHOLD %s.Only decimal or hex format is allowed.' % 
Threshold_Str)
+return Threshold
+
+def CheckFreeSizeThreshold(self, Threshold=None, FvDir=None):
+if not isinstance(Threshold, int):
+return
+if not isinstance(FvDir, str) or not FvDir:
+return
+FdfParserObject = GlobalData.gFdfParser
+FvRegionNameList = [FvName for FvName in 
FdfParserObject.Profile.FvDict if 
FdfParserObject.Profile.FvDict[FvName].FvRegionInFD]
+for FvName in FdfParserObject.Profile.FvDict:
+if FvName in FvRegionNameList:
+FvSpaceInfoFileName = os.path.join(FvDir, FvName.upper() + 
'.Fv.map')
+if os.path.exists(FvSpaceInfoFileName):
+FileLinesList = getlines(FvSpaceInfoFileName)
+for Line in FileLinesList:
+NameValue = Line.split('=')
+if len(NameValue) == 2 and NameValue[0].strip() == 
'EFI_FV_SPACE_SIZE':
+FreeSizeValue = int(NameValue[1].strip(), 0)
+if FreeSizeValue < Threshold:
+EdkLogger.error("build", FV_FREESIZE_ERROR,
+'the required spare space in 
fv image size %d of %s FV exceeds the set spare space in fv image size %d' % (
+FreeSizeValue, FvName, 

[edk2-devel] [PATCH V4] BaseTools:Add the spare space FV image size checker

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654

If FV is placed in FD region, its FV image size is fixed.
When FV image size exceeds it, it will trig the build break.
To alert the developer to adjust FV image size earlier,
I request to add new checker for the the spare FV space.
When the spare FV space is less than the specified threshold,
build tool will report the error.

This checker is the optional.
It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=1.
Macro is the value of the spare space threshold size.
It can be decimal or hex format. If it is enabled,
BaseTools will check every FV with the fixed size.
If FV doesn't meet with the size requirement,
Build tool will report error message to say there is no
enough spare space.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 Determine the value of Threshold and FvDir

 BaseTools/Source/Python/Common/BuildToolError.py |  2 +
 BaseTools/Source/Python/build/build.py   | 47 ++
 2 files changed, 49 insertions(+)

diff --git a/BaseTools/Source/Python/Common/BuildToolError.py 
b/BaseTools/Source/Python/Common/BuildToolError.py
index ecc83d0f48..21549683cd 100644
--- a/BaseTools/Source/Python/Common/BuildToolError.py
+++ b/BaseTools/Source/Python/Common/BuildToolError.py
@@ -64,6 +64,8 @@ COMMAND_FAILURE = 0x7000
 
 PERMISSION_FAILURE = 0x8000
 
+FV_FREESIZE_ERROR = 0x9000
+
 CODE_ERROR = 0xC0DE
 
 AUTOGEN_ERROR = 0xF000
diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index bec848a7b2..c19be67254 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -25,6 +25,7 @@ import traceback
 import multiprocessing
 from threading import Thread,Event,BoundedSemaphore
 import threading
+from linecache import getlines
 from subprocess import Popen,PIPE, STDOUT
 from collections import OrderedDict, defaultdict
 
@@ -1413,6 +1414,9 @@ class Build():
 if Target == 'fds':
 if GenFdsApi(AutoGenObject.GenFdsCommandDict, self.Db):
 EdkLogger.error("build", COMMAND_FAILURE)
+Threshold = self.GetFreeSizeThreshold()
+if Threshold:
+self.CheckFreeSizeThreshold(Threshold, AutoGenObject.FvDir)
 return True
 
 # run
@@ -2311,6 +2315,9 @@ class Build():
 GenFdsStart = time.time()
 if GenFdsApi(Wa.GenFdsCommandDict, self.Db):
 EdkLogger.error("build", COMMAND_FAILURE)
+Threshold = self.GetFreeSizeThreshold()
+if Threshold:
+self.CheckFreeSizeThreshold(Threshold, Wa.FvDir)
 
 #
 # Create MAP file for all platform FVs after GenFds.
@@ -2322,6 +2329,46 @@ class Build():
 #
 self._SaveMapFile(MapBuffer, Wa)
 self.CreateGuidedSectionToolsFile(Wa)
+
+## GetFreeSizeThreshold()
+#
+#   @retval int Threshold value
+#
+def GetFreeSizeThreshold(self):
+Threshold = None
+Threshold_Str = 
GlobalData.gCommandLineDefines.get('FV_SPARE_SPACE_THRESHOLD')
+if Threshold_Str:
+try:
+if Threshold_Str.lower().startswith('0x'):
+Threshold = int(Threshold_Str, 16)
+else:
+Threshold = int(Threshold_Str)
+except:
+EdkLogger.warn("build", 'incorrect value for 
FV_SPARE_SPACE_THRESHOLD %s. It can be decimal or hex format' % Threshold_Str)
+return Threshold
+
+def CheckFreeSizeThreshold(self, Threshold=None, FvDir=None):
+if not isinstance(Threshold, int):
+return
+if not isinstance(FvDir, str) or not FvDir:
+return
+FdfParserObject = GlobalData.gFdfParser
+FvRegionNameList = [FvName for FvName in 
FdfParserObject.Profile.FvDict if 
FdfParserObject.Profile.FvDict[FvName].FvRegionInFD]
+for FvName in FdfParserObject.Profile.FvDict:
+if FvName in FvRegionNameList:
+FvSpaceInfoFileName = os.path.join(FvDir, FvName.upper() + 
'.Fv.map')
+if os.path.exists(FvSpaceInfoFileName):
+FileLinesList = getlines(FvSpaceInfoFileName)
+for Line in FileLinesList:
+NameValue = Line.split('=')
+if len(NameValue) == 2 and NameValue[0].strip() == 
'EFI_FV_SPACE_SIZE':
+FreeSizeValue = int(NameValue[1].strip(), 0)
+if FreeSizeValue < Threshold:
+EdkLogger.error("build", FV_FREESIZE_ERROR,
+'Freespace value %d of %s FV 
is smaller than threshold, The value of FV_SPARE_SPACE_THRESHOLD is %d' % (
+FreeSizeValue, 

[edk2-devel] [PATCH V3] BaseTools:Add the spare space FV image size checker

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654

If FV is placed in FD region, its FV image size is fixed.
When FV image size exceeds it, it will trig the build break.
To alert the developer to adjust FV image size earlier,
I request to add new checker for the the spare FV space.
When the spare FV space is less than the specified threshold,
build tool will report the error.

This checker is the optional.
It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=1.
Macro is the value of the spare space threshold size.
It can be decimal or hex format. If it is enabled,
BaseTools will check every FV with the fixed size.
If FV doesn't meet with the size requirement,
Build tool will report error message to say there is no
enough spare space.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 move the functions into build.py from GenFds.py

 BaseTools/Source/Python/Common/BuildToolError.py |  2 ++
 BaseTools/Source/Python/build/build.py   | 44 +++
 2 files changed, 46 insertions(+)

diff --git a/BaseTools/Source/Python/Common/BuildToolError.py 
b/BaseTools/Source/Python/Common/BuildToolError.py
index ecc83d0f48bd..21549683cd19 100644
--- a/BaseTools/Source/Python/Common/BuildToolError.py
+++ b/BaseTools/Source/Python/Common/BuildToolError.py
@@ -64,6 +64,8 @@ COMMAND_FAILURE = 0x7000
 
 PERMISSION_FAILURE = 0x8000
 
+FV_FREESIZE_ERROR = 0x9000
+
 CODE_ERROR = 0xC0DE
 
 AUTOGEN_ERROR = 0xF000
diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index bec848a7b2e3..3faa11199f5f 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -25,6 +25,7 @@ import traceback
 import multiprocessing
 from threading import Thread,Event,BoundedSemaphore
 import threading
+from linecache import getlines
 from subprocess import Popen,PIPE, STDOUT
 from collections import OrderedDict, defaultdict
 
@@ -61,6 +62,7 @@ from AutoGen.ModuleAutoGenHelper import WorkSpaceInfo, 
PlatformInfo
 from GenFds.FdfParser import FdfParser
 from AutoGen.IncludesAutoGen import IncludesAutoGen
 from GenFds.GenFds import resetFdsGlobalVariable
+from GenFds.GenFdsGlobalVariable import GenFdsGlobalVariable
 
 ## standard targets of build command
 gSupportedTarget = ['all', 'genc', 'genmake', 'modules', 'libraries', 'fds', 
'clean', 'cleanall', 'cleanlib', 'run']
@@ -1413,6 +1415,9 @@ class Build():
 if Target == 'fds':
 if GenFdsApi(AutoGenObject.GenFdsCommandDict, self.Db):
 EdkLogger.error("build", COMMAND_FAILURE)
+Threshold = self.GetFreeSizeThreshold()
+if Threshold:
+self.CheckFreeSizeThreshold(Threshold, AutoGenObject.FvDir)
 return True
 
 # run
@@ -2311,6 +2316,9 @@ class Build():
 GenFdsStart = time.time()
 if GenFdsApi(Wa.GenFdsCommandDict, self.Db):
 EdkLogger.error("build", COMMAND_FAILURE)
+Threshold = self.GetFreeSizeThreshold()
+if Threshold:
+self.CheckFreeSizeThreshold(Threshold, Wa.FvDir)
 
 #
 # Create MAP file for all platform FVs after GenFds.
@@ -2322,6 +2330,42 @@ class Build():
 #
 self._SaveMapFile(MapBuffer, Wa)
 self.CreateGuidedSectionToolsFile(Wa)
+
+## GetFreeSizeThreshold()
+#
+#   @retval int Threshold value
+#
+def GetFreeSizeThreshold(self):
+Threshold = None
+Threshold_Str = 
GlobalData.gCommandLineDefines.get('FV_SPARE_SPACE_THRESHOLD')
+if Threshold_Str:
+try:
+if Threshold_Str.lower().startswith('0x'):
+Threshold = int(Threshold_Str, 16)
+else:
+Threshold = int(Threshold_Str)
+except:
+EdkLogger.warn("build", 'incorrect value for 
FV_SPARE_SPACE_THRESHOLD %s. It can be decimal or hex format' % Threshold_Str)
+return Threshold
+
+## CheckFreeSizeThreshold()
+def CheckFreeSizeThreshold(self, Threshold=None, FvDir=None):
+FdfParserObject = GlobalData.gFdfParser
+FvRegionNameList = [FvName for FvName in 
FdfParserObject.Profile.FvDict if 
FdfParserObject.Profile.FvDict[FvName].FvRegionInFD]
+for FvName in FdfParserObject.Profile.FvDict:
+if FvName in FvRegionNameList:
+FvSpaceInfoFileName = os.path.join(FvDir, FvName.upper() + 
'.Fv.map')
+if os.path.exists(FvSpaceInfoFileName):
+FileLinesList = getlines(FvSpaceInfoFileName)
+for Line in FileLinesList:
+NameValue = Line.split('=')
+if len(NameValue) == 2 and NameValue[0].strip() == 
'EFI_FV_SPACE_SIZE':
+FreeSizeValue = int(NameValue[1].strip(), 0)

[edk2-devel] [PATCH V2] BaseTools:Add the spare space FV image size checker

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654

If FV is placed in FD region, its FV image size is fixed.
When FV image size exceeds it, it will trig the build break.
To alert the developer to adjust FV image size earlier,
I request to add new checker for the the spare FV space.
When the spare FV space is less than the specified threshold,
build tool will report the error.

This checker is the optional.
It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=1.
Macro is the value of the spare space threshold size.
It can be decimal or hex format. If it is enabled,
BaseTools will check every FV with the fixed size.
If FV doesn't meet with the size requirement,
Build tool will report error message to say there is no
enough spare space.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 move the functions into build.py from GenFds.py

 BaseTools/Source/Python/Common/BuildToolError.py |  2 ++
 BaseTools/Source/Python/build/build.py   | 43 +++
 2 files changed, 45 insertions(+)

diff --git a/BaseTools/Source/Python/Common/BuildToolError.py 
b/BaseTools/Source/Python/Common/BuildToolError.py
index ecc83d0f48bd..21549683cd19 100644
--- a/BaseTools/Source/Python/Common/BuildToolError.py
+++ b/BaseTools/Source/Python/Common/BuildToolError.py
@@ -64,6 +64,8 @@ COMMAND_FAILURE = 0x7000
 
 PERMISSION_FAILURE = 0x8000
 
+FV_FREESIZE_ERROR = 0x9000
+
 CODE_ERROR = 0xC0DE
 
 AUTOGEN_ERROR = 0xF000
diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index bec848a7b2e3..b95d91e35390 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -25,6 +25,7 @@ import traceback
 import multiprocessing
 from threading import Thread,Event,BoundedSemaphore
 import threading
+from linecache import getlines
 from subprocess import Popen,PIPE, STDOUT
 from collections import OrderedDict, defaultdict
 
@@ -61,6 +62,7 @@ from AutoGen.ModuleAutoGenHelper import WorkSpaceInfo, 
PlatformInfo
 from GenFds.FdfParser import FdfParser
 from AutoGen.IncludesAutoGen import IncludesAutoGen
 from GenFds.GenFds import resetFdsGlobalVariable
+from GenFds.GenFdsGlobalVariable import GenFdsGlobalVariable
 
 ## standard targets of build command
 gSupportedTarget = ['all', 'genc', 'genmake', 'modules', 'libraries', 'fds', 
'clean', 'cleanall', 'cleanlib', 'run']
@@ -1413,6 +1415,9 @@ class Build():
 if Target == 'fds':
 if GenFdsApi(AutoGenObject.GenFdsCommandDict, self.Db):
 EdkLogger.error("build", COMMAND_FAILURE)
+Threshold = self.GetFreeSizeThreshold()
+if Threshold:
+self.CheckFreeSizeThreshold(Threshold, AutoGenObject.FvDir)
 return True
 
 # run
@@ -2311,6 +2316,9 @@ class Build():
 GenFdsStart = time.time()
 if GenFdsApi(Wa.GenFdsCommandDict, self.Db):
 EdkLogger.error("build", COMMAND_FAILURE)
+Threshold = self.GetFreeSizeThreshold()
+if Threshold:
+self.CheckFreeSizeThreshold(Threshold, Wa.FvDir)
 
 #
 # Create MAP file for all platform FVs after GenFds.
@@ -2322,6 +2330,41 @@ class Build():
 #
 self._SaveMapFile(MapBuffer, Wa)
 self.CreateGuidedSectionToolsFile(Wa)
+
+## GetFreeSizeThreshold()
+#
+#   @retval int Threshold value
+#
+def GetFreeSizeThreshold(self):
+Threshold = None
+Threshold_Str = 
GlobalData.gCommandLineDefines.get('FV_SPARE_SPACE_THRESHOLD')
+if Threshold_Str:
+try:
+if Threshold_Str.lower().startswith('0x'):
+Threshold = int(Threshold_Str, 16)
+else:
+Threshold = int(Threshold_Str)
+except:
+EdkLogger.warn("build", 'incorrect value for 
FV_SPARE_SPACE_THRESHOLD %s. It can be decimal or hex format' % Threshold_Str)
+return Threshold
+
+## CheckFreeSizeThreshold()
+def CheckFreeSizeThreshold(self, Threshold=None, FvDir=None):
+FdfParserObject = GlobalData.gFdfParser
+FvRegionNameList = [FvName for FvName in 
FdfParserObject.Profile.FvDict if 
FdfParserObject.Profile.FvDict[FvName].FvRegionInFD]
+for FvName in FdfParserObject.Profile.FvDict:
+FvSpaceInfoFileName = os.path.join(FvDir, FvName.upper() + 
'.Fv.map')
+if os.path.exists(FvSpaceInfoFileName):
+FileLinesList = getlines(FvSpaceInfoFileName)
+for Line in FileLinesList:
+NameValue = Line.split('=')
+if len(NameValue) == 2 and NameValue[0].strip() == 
'EFI_FV_SPACE_SIZE':
+FreeSizeValue = int(NameValue[1].strip(), 0)
+if FvName in FvRegionNameList and 

Re: [edk2-devel] [PATCH 1/1] BaseTools:Add the spare space FV image size checker

[Fan, ZhijuX]

Hi,

It can be implemented independent of the GenFds tool
It accepts the parameters passed by building and GenFds Tool

Best Regards
Fan Zhiju



From: sean.brogan via [] 
Sent: Friday, April 3, 2020 10:51 AM
To: Fan; Fan, ZhijuX ; devel@edk2.groups.io
Subject: Re: [edk2-devel] [PATCH 1/1] BaseTools:Add the spare space FV image 
size checker

I would have much rather seen this implemented independent of the GenFds tool 
as this is a nice feature but shouldn't be tightly coupled into the tool that 
does building and packaging firmware.

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#56924): https://edk2.groups.io/g/devel/message/56924
Mute This Topic: https://groups.io/mt/72740912/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH 1/1] BaseTools:Add the spare space FV image size checker

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654

If FV is placed in FD region, its FV image size is fixed.
When FV image size exceeds it, it will trig the build break.
To alert the developer to adjust FV image size earlier,
I request to add new checker for the the spare FV space.
When the spare FV space is less than the specified threshold,
build tool will report the error.

This checker is the optional.
It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=1.
Macro is the value of the spare space threshold size.
It can be decimal or hex format. If it is enabled,
BaseTools will check every FV with the fixed size.
If FV doesn't meet with the size requirement,
Build tool will report error message to say there is no
enough spare space.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Common/BuildToolError.py |  2 ++
 BaseTools/Source/Python/GenFds/GenFds.py | 32 +---
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/BaseTools/Source/Python/Common/BuildToolError.py 
b/BaseTools/Source/Python/Common/BuildToolError.py
index ecc83d0f48bd..21549683cd19 100644
--- a/BaseTools/Source/Python/Common/BuildToolError.py
+++ b/BaseTools/Source/Python/Common/BuildToolError.py
@@ -64,6 +64,8 @@ COMMAND_FAILURE = 0x7000
 
 PERMISSION_FAILURE = 0x8000
 
+FV_FREESIZE_ERROR = 0x9000
+
 CODE_ERROR = 0xC0DE
 
 AUTOGEN_ERROR = 0xF000
diff --git a/BaseTools/Source/Python/GenFds/GenFds.py 
b/BaseTools/Source/Python/GenFds/GenFds.py
index d5511f4c40e5..c4782e89111b 100644
--- a/BaseTools/Source/Python/GenFds/GenFds.py
+++ b/BaseTools/Source/Python/GenFds/GenFds.py
@@ -29,7 +29,7 @@ from Common.Misc import DirCache, PathClass, 
GuidStructureStringToGuidString
 from Common.Misc import SaveFileOnChange, ClearDuplicatedInf
 from Common.BuildVersion import gBUILD_VERSION
 from Common.MultipleWorkspace import MultipleWorkspace as mws
-from Common.BuildToolError import FatalError, GENFDS_ERROR, CODE_ERROR, 
FORMAT_INVALID, RESOURCE_NOT_AVAILABLE, FILE_NOT_FOUND, OPTION_MISSING, 
FORMAT_NOT_SUPPORTED, OPTION_VALUE_INVALID, PARAMETER_INVALID
+from Common.BuildToolError import FatalError, GENFDS_ERROR, CODE_ERROR, 
FORMAT_INVALID, RESOURCE_NOT_AVAILABLE, FILE_NOT_FOUND, OPTION_MISSING, 
FORMAT_NOT_SUPPORTED, OPTION_VALUE_INVALID, PARAMETER_INVALID, FV_FREESIZE_ERROR
 from Workspace.WorkspaceDatabase import WorkspaceDatabase
 
 from .FdfParser import FdfParser, Warning
@@ -374,7 +374,8 @@ def GenFdsApi(FdsCommandDict, WorkSpaceDataBase=None):
 GenFds.GenerateGuidXRefFile(BuildWorkSpace, ArchList, FdfParserObj)
 
 """Display FV space info."""
-GenFds.DisplayFvSpaceInfo(FdfParserObj)
+Threshold = GenFds.GetFreeSizeThreshold()
+GenFds.DisplayFvSpaceInfo(FdfParserObj, Threshold)
 
 except Warning as X:
 EdkLogger.error(X.ToolName, FORMAT_INVALID, File=X.FileName, 
Line=X.LineNumber, ExtraData=X.Message, RaiseError=False)
@@ -584,13 +585,31 @@ class GenFds(object):
 return 
ElementRegion.BlockSizeOfRegion(ElementFd.BlockSizeList)
 return DefaultBlockSize
 
+## GetFreeSizeThreshold()
+#
+#   @retval int Threshold value
+#
+@staticmethod
+def GetFreeSizeThreshold():
+Threshold = None
+Threshold_Str = 
GlobalData.gCommandLineDefines.get('FV_SPARE_SPACE_THRESHOLD')
+if Threshold_Str:
+try:
+if Threshold_Str.lower().startswith('0x'):
+Threshold = int(Threshold_Str, 16)
+else:
+Threshold = int(Threshold_Str)
+except:
+EdkLogger.warn("GenFds", 'incorrect value for 
FV_SPARE_SPACE_THRESHOLD %s. It can be decimal or hex format' % Threshold_Str)
+return Threshold
+
 ## DisplayFvSpaceInfo()
 #
 #   @param  FvObj   Whose block size to get
 #   @retval None
 #
 @staticmethod
-def DisplayFvSpaceInfo(FdfParserObject):
+def DisplayFvSpaceInfo(FdfParserObject, Threshold):
 
 FvSpaceInfoList = []
 MaxFvNameLength = 0
@@ -623,6 +642,10 @@ class GenFds(object):
 FvSpaceInfoList.append((FvName, Total, Used, Free))
 
 GenFdsGlobalVariable.InfLogger('\nFV Space Information')
+if Threshold:
+FvRegionNameList = [FvName for FvName in 
FdfParserObject.Profile.FvDict if 
FdfParserObject.Profile.FvDict[FvName].FvRegionInFD]
+else:
+FvRegionNameList = []
 for FvSpaceInfo in FvSpaceInfoList:
 Name = FvSpaceInfo[0]
 TotalSizeValue = int(FvSpaceInfo[1], 0)
@@ -634,6 +657,9 @@ class GenFds(object):
 Percentage = str((UsedSizeValue + 0.0) / 
TotalSizeValue)[0:4].lstrip('0.')
 
 GenFdsGlobalVariable.InfLogger(Name + ' ' + '[' + Percentage + 
'%Full] ' + str(TotalSizeValue) + ' total, ' + str(UsedSizeValue) + ' used, ' + 
str(FreeSizeValue) + ' 

[edk2-devel] [PATCH V6] BaseTools:GuidedSectionTools.txt is not generated correctly

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538

For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.

*_*_*_LZMA_PATH = LzmaCompressPlatform

This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.

Signed-off-by: Zhiju.Fan 
Cc: Liming Gao 
Cc: Bob Feng 
---
Some unnecessary comments were removed

 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py |  3 ++-
 BaseTools/Source/Python/build/build.py | 16 +++-
 2 files changed, 5 insertions(+), 14 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py 
b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
index d32178b00c93..af66c48c7d6a 100644
--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
@@ -866,7 +866,8 @@ class PlatformAutoGen(AutoGen):
 Value += " " + 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
 else:
 Value = 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
-
+Def = '_'.join([self.BuildTarget, self.ToolChain, 
self.Arch, Tool, Attr])
+self.Workspace.ToolDef.ToolsDefTxtDictionary[Def] 
= Value
 if Attr == "PATH":
 # Don't put MAKE definition in the file
 if Tool != "MAKE":
diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index d841fefdc502..bec848a7b2e3 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -2347,7 +2347,7 @@ class Build():
 toolName = split[3]
 path = '_'.join(split[0:4]) + '_PATH'
 path = self.ToolDef.ToolsDefTxtDictionary[path]
-path = self.GetFullPathOfTool(path)
+path = self.GetRealPathOfTool(path)
 guidAttribs.append((guid, toolName, path))
 
 # Write out GuidedSecTools.txt
@@ -2357,21 +2357,11 @@ class Build():
 print(' '.join(guidedSectionTool), file=toolsFile)
 toolsFile.close()
 
-## Returns the full path of the tool.
+## Returns the real path of the tool.
 #
-def GetFullPathOfTool (self, tool):
+def GetRealPathOfTool (self, tool):
 if os.path.exists(tool):
 return os.path.realpath(tool)
-else:
-# We need to search for the tool using the
-# PATH environment variable.
-for dirInPath in os.environ['PATH'].split(os.pathsep):
-foundPath = os.path.join(dirInPath, tool)
-if os.path.exists(foundPath):
-return os.path.realpath(foundPath)
-
-# If the tool was not found in the path then we just return
-# the input tool.
 return tool
 
 ## Launch the module or platform build
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#56384): https://edk2.groups.io/g/devel/message/56384
Mute This Topic: https://groups.io/mt/72560484/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH V5] BaseTools:GuidedSectionTools.txt is not generated correctly

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538

For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.

*_*_*_LZMA_PATH = LzmaCompressPlatform

This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.

Signed-off-by: Zhiju.Fan 
Cc: Liming Gao 
Cc: Bob Feng 
---

A judgment condition was removed
if self.Workspace.ToolDef.ToolsDefTxtDictionary.get(Def):

 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py |  3 ++-
 BaseTools/Source/Python/build/build.py | 13 +++--
 2 files changed, 5 insertions(+), 11 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py 
b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
index d32178b00c93..af66c48c7d6a 100644
--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
@@ -866,7 +866,8 @@ class PlatformAutoGen(AutoGen):
 Value += " " + 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
 else:
 Value = 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
-
+Def = '_'.join([self.BuildTarget, self.ToolChain, 
self.Arch, Tool, Attr])
+self.Workspace.ToolDef.ToolsDefTxtDictionary[Def] 
= Value
 if Attr == "PATH":
 # Don't put MAKE definition in the file
 if Tool != "MAKE":
diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index d841fefdc502..6dbe05f0a93e 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -2363,16 +2363,9 @@ class Build():
 if os.path.exists(tool):
 return os.path.realpath(tool)
 else:
-# We need to search for the tool using the
-# PATH environment variable.
-for dirInPath in os.environ['PATH'].split(os.pathsep):
-foundPath = os.path.join(dirInPath, tool)
-if os.path.exists(foundPath):
-return os.path.realpath(foundPath)
-
-# If the tool was not found in the path then we just return
-# the input tool.
-return tool
+# If the tool was not found in the path then we just return
+# the input tool.
+return tool
 
 ## Launch the module or platform build
 #
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#56377): https://edk2.groups.io/g/devel/message/56377
Mute This Topic: https://groups.io/mt/72559525/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH 1/1] BaseTools:Fix build tools print traceback info issue

[Fan, ZhijuX]

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2610

We meet a case that the DEC file declaring the PCD isn't
included in the INF.it cause build tools report Traceback error.

Remove raise statements that generate Tracebacks that were only
intended for development/debug. With the raise statements removed
proper error messages are shown.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/AutoGenWorker.py | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py 
b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py
index 40b448f5b2db..563d91b421ce 100755
--- a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py
@@ -256,7 +256,6 @@ class AutoGenWorkerInProcess(mp.Process):
 CacheResult = Ma.CanSkipbyPreMakeCache()
 except:
 CacheResult = False
-traceback.print_exc(file=sys.stdout)
 self.feedback_q.put(taskname)
 
 if CacheResult:
@@ -273,7 +272,6 @@ class AutoGenWorkerInProcess(mp.Process):
 CacheResult = Ma.CanSkipbyMakeCache()
 except:
 CacheResult = False
-traceback.print_exc(file=sys.stdout)
 self.feedback_q.put(taskname)
 
 if CacheResult:
@@ -285,7 +283,6 @@ class AutoGenWorkerInProcess(mp.Process):
 except Empty:
 pass
 except:
-traceback.print_exc(file=sys.stdout)
 self.feedback_q.put(taskname)
 finally:
 self.feedback_q.put("Done")
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#56047): https://edk2.groups.io/g/devel/message/56047
Mute This Topic: https://groups.io/mt/72093144/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH V4] BaseTools:GuidedSectionTools.txt is not generated correctly

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538

For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.

*_*_*_LZMA_PATH = LzmaCompressPlatform

This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.

Signed-off-by: Zhiju.Fan 
Cc: Liming Gao 
Cc: Bob Feng 
---
 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py |  4 +++-
 BaseTools/Source/Python/build/build.py | 17 -
 2 files changed, 3 insertions(+), 18 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py 
b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
index d32178b00c93..478a5116fd7a 100644
--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
@@ -866,7 +866,9 @@ class PlatformAutoGen(AutoGen):
 Value += " " + 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
 else:
 Value = 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
-
+Def = '_'.join([self.BuildTarget, self.ToolChain, 
self.Arch, Tool, Attr])
+if 
self.Workspace.ToolDef.ToolsDefTxtDictionary.get(Def):
+
self.Workspace.ToolDef.ToolsDefTxtDictionary[Def] = Value
 if Attr == "PATH":
 # Don't put MAKE definition in the file
 if Tool != "MAKE":
diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index d841fefdc502..487c57d6e7ff 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -2347,7 +2347,6 @@ class Build():
 toolName = split[3]
 path = '_'.join(split[0:4]) + '_PATH'
 path = self.ToolDef.ToolsDefTxtDictionary[path]
-path = self.GetFullPathOfTool(path)
 guidAttribs.append((guid, toolName, path))
 
 # Write out GuidedSecTools.txt
@@ -2357,22 +2356,6 @@ class Build():
 print(' '.join(guidedSectionTool), file=toolsFile)
 toolsFile.close()
 
-## Returns the full path of the tool.
-#
-def GetFullPathOfTool (self, tool):
-if os.path.exists(tool):
-return os.path.realpath(tool)
-else:
-# We need to search for the tool using the
-# PATH environment variable.
-for dirInPath in os.environ['PATH'].split(os.pathsep):
-foundPath = os.path.join(dirInPath, tool)
-if os.path.exists(foundPath):
-return os.path.realpath(foundPath)
-
-# If the tool was not found in the path then we just return
-# the input tool.
-return tool
 
 ## Launch the module or platform build
 #
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#55948): https://edk2.groups.io/g/devel/message/55948
Mute This Topic: https://groups.io/mt/72041911/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH V3] BaseTools:GuidedSectionTools.txt is not generated correctly

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538

For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.

*_*_*_LZMA_PATH = LzmaCompressPlatform

This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.

Signed-off-by: Zhiju.Fan 
Cc: Liming Gao 
Cc: Bob Feng 
---
Change full path is only used on non-windows systems

 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py | 4 +++-
 BaseTools/Source/Python/build/build.py | 3 ++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py 
b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
index d32178b00c93..478a5116fd7a 100644
--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
@@ -866,7 +866,9 @@ class PlatformAutoGen(AutoGen):
 Value += " " + 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
 else:
 Value = 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
-
+Def = '_'.join([self.BuildTarget, self.ToolChain, 
self.Arch, Tool, Attr])
+if 
self.Workspace.ToolDef.ToolsDefTxtDictionary.get(Def):
+
self.Workspace.ToolDef.ToolsDefTxtDictionary[Def] = Value
 if Attr == "PATH":
 # Don't put MAKE definition in the file
 if Tool != "MAKE":
diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index d841fefdc502..4d4615c0e94a 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -2347,7 +2347,8 @@ class Build():
 toolName = split[3]
 path = '_'.join(split[0:4]) + '_PATH'
 path = self.ToolDef.ToolsDefTxtDictionary[path]
-path = self.GetFullPathOfTool(path)
+if sys.platform != "win32":
+path = self.GetFullPathOfTool(path)
 guidAttribs.append((guid, toolName, path))
 
 # Write out GuidedSecTools.txt
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#55927): https://edk2.groups.io/g/devel/message/55927
Mute This Topic: https://groups.io/mt/72020926/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH V4] BaseTools:copy the common PcdValueCommon.c to output directory

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2568

PcdValueInit shares the same Edk2\BaseTools\Source\C\PcdValueCommon.c.
To avoid the conflict, it should copy this file to its output directory,
If so, PcdValueCommon.c file will be private for PcdValueInit

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
Optimized the generated Makefile and script code
'%s' % PcdValueCommonName change to PcdValueCommonName

 BaseTools/Source/Python/Workspace/DscBuildData.py | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py 
b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 476c7edaf9da..75f419c7ff1b 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -55,6 +55,7 @@ def _IsFieldValueAnArray (Value):
 return False
 
 PcdValueInitName = 'PcdValueInit'
+PcdValueCommonName = 'PcdValueCommon'
 
 PcdMainCHeader = '''
 /**
@@ -2634,10 +2635,10 @@ class DscBuildData(PlatformBuildClassObject):
 
 MakeApp = PcdMakefileHeader
 if sys.platform == "win32":
-MakeApp = MakeApp + 'APPFILE = %s\%s.exe\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = 
%s\%s.obj %s.obj\n' % (self.OutputPath, PcdValueInitName, 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon"))) + 'INC = '
+MakeApp = MakeApp + 'APPFILE = %s\%s.exe\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = 
%s\%s.obj %s.obj\n' % (self.OutputPath, PcdValueInitName, 
os.path.join(self.OutputPath, PcdValueCommonName)) + 'INC = '
 else:
 MakeApp = MakeApp + PcdGccMakefile
-MakeApp = MakeApp + 'APPFILE = %s/%s\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = %s/%s.o 
%s.o\n' % (self.OutputPath, PcdValueInitName, 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon"))) + \
+MakeApp = MakeApp + 'APPFILE = %s/%s\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = %s/%s.o 
%s.o\n' % (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath, 
PcdValueCommonName)) + \
   'include $(MAKEROOT)/Makefiles/app.makefile\n' + 
'INCLUDE +='
 
 IncSearchList = []
@@ -2742,6 +2743,14 @@ class DscBuildData(PlatformBuildClassObject):
 IncFileList = GetDependencyList(IncludeFileFullPaths, SearchPathList)
 for include_file in IncFileList:
 MakeApp += "$(OBJECTS) : %s\n" % include_file
+if sys.platform == "win32":
+PcdValueCommonPath = 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source\C\Common\PcdValueCommon.c"))
+MakeApp = MakeApp + '%s\PcdValueCommon.c : %s\n' % 
(self.OutputPath, PcdValueCommonPath)
+MakeApp = MakeApp + '\tcopy /y %s $@\n' % (PcdValueCommonPath)
+else:
+PcdValueCommonPath = 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon.c"))
+MakeApp = MakeApp + '%s/PcdValueCommon.c : %s\n' % 
(self.OutputPath, PcdValueCommonPath)
+MakeApp = MakeApp + '\tcp -f %s %s/PcdValueCommon.c\n' % 
(PcdValueCommonPath, self.OutputPath)
 MakeFileName = os.path.join(self.OutputPath, 'Makefile')
 MakeApp += "$(OBJECTS) : %s\n" % MakeFileName
 SaveFileOnChange(MakeFileName, MakeApp, False)
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#55747): https://edk2.groups.io/g/devel/message/55747
Mute This Topic: https://groups.io/mt/71875979/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH V3] BaseTools:copy the common PcdValueCommon.c to output directory

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2568

PcdValueInit shares the same Edk2\BaseTools\Source\C\PcdValueCommon.c.
To avoid the conflict, it should copy this file to its output directory,
If so, PcdValueCommon.c file will be private for PcdValueInit

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
Optimized the generated Makefile

 BaseTools/Source/Python/Workspace/DscBuildData.py | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py 
b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 476c7edaf9da..75f419c7ff1b 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -55,6 +55,7 @@ def _IsFieldValueAnArray (Value):
 return False
 
 PcdValueInitName = 'PcdValueInit'
+PcdValueCommonName = 'PcdValueCommon'
 
 PcdMainCHeader = '''
 /**
@@ -2634,10 +2635,10 @@ class DscBuildData(PlatformBuildClassObject):
 
 MakeApp = PcdMakefileHeader
 if sys.platform == "win32":
-MakeApp = MakeApp + 'APPFILE = %s\%s.exe\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = 
%s\%s.obj %s.obj\n' % (self.OutputPath, PcdValueInitName, 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon"))) + 'INC = '
+MakeApp = MakeApp + 'APPFILE = %s\%s.exe\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = 
%s\%s.obj %s.obj\n' % (self.OutputPath, PcdValueInitName, 
os.path.join(self.OutputPath,'%s' % PcdValueCommonName)) + 'INC = '
 else:
 MakeApp = MakeApp + PcdGccMakefile
-MakeApp = MakeApp + 'APPFILE = %s/%s\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = %s/%s.o 
%s.o\n' % (self.OutputPath, PcdValueInitName, 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon"))) + \
+MakeApp = MakeApp + 'APPFILE = %s/%s\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = %s/%s.o 
%s.o\n' % (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,'%s' 
% PcdValueCommonName)) + \
   'include $(MAKEROOT)/Makefiles/app.makefile\n' + 
'INCLUDE +='
 
 IncSearchList = []
@@ -2742,6 +2743,14 @@ class DscBuildData(PlatformBuildClassObject):
 IncFileList = GetDependencyList(IncludeFileFullPaths, SearchPathList)
 for include_file in IncFileList:
 MakeApp += "$(OBJECTS) : %s\n" % include_file
+if sys.platform == "win32":
+PcdValueCommonPath = 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source\C\Common\PcdValueCommon.c"))
+MakeApp = MakeApp + '%s\PcdValueCommon.c : %s\n' % 
(self.OutputPath, PcdValueCommonPath)
+MakeApp = MakeApp + '\tcopy /y %s $@\n' % (PcdValueCommonPath)
+else:
+PcdValueCommonPath = 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon.c"))
+MakeApp = MakeApp + '%s/PcdValueCommon.c : %s\n' % 
(self.OutputPath, PcdValueCommonPath)
+MakeApp = MakeApp + '\tcp -f %s %s/PcdValueCommon.c\n' % 
(PcdValueCommonPath, self.OutputPath)
 MakeFileName = os.path.join(self.OutputPath, 'Makefile')
 MakeApp += "$(OBJECTS) : %s\n" % MakeFileName
 SaveFileOnChange(MakeFileName, MakeApp, False)
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#55745): https://edk2.groups.io/g/devel/message/55745
Mute This Topic: https://groups.io/mt/71875357/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH V2] BaseTools:copy the common PcdValueCommon.c to output directory

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2568

PcdValueInit shares the same Edk2\BaseTools\Source\C\PcdValueCommon.c.
To avoid the conflict, it should copy this file to its output directory,
If so, PcdValueCommon.c file will be private for PcdValueInit

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
copy source file PcdValueCommon.c in the generated Makefile instead
of do it in python script

 BaseTools/Source/Python/Workspace/DscBuildData.py | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py 
b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 476c7edaf9da..07647e835875 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -55,6 +55,7 @@ def _IsFieldValueAnArray (Value):
 return False
 
 PcdValueInitName = 'PcdValueInit'
+PcdValueCommonName = 'PcdValueCommon'
 
 PcdMainCHeader = '''
 /**
@@ -2634,10 +2635,18 @@ class DscBuildData(PlatformBuildClassObject):
 
 MakeApp = PcdMakefileHeader
 if sys.platform == "win32":
-MakeApp = MakeApp + 'APPFILE = %s\%s.exe\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = 
%s\%s.obj %s.obj\n' % (self.OutputPath, PcdValueInitName, 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon"))) + 'INC = '
+PcdValueCommonPath = 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source\C\Common\PcdValueCommon.c"))
+MakeApp = MakeApp + 'all:%s\PcdValueCommon.c\n' % self.OutputPath
+MakeApp = MakeApp + '%s\PcdValueCommon.c : %s\n' % 
(self.OutputPath, PcdValueCommonPath)
+MakeApp = MakeApp + '\tcopy /y %s $@\n' % (PcdValueCommonPath)
+MakeApp = MakeApp + 'APPFILE = %s\%s.exe\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = 
%s\%s.obj %s.obj\n' % (self.OutputPath, PcdValueInitName, 
os.path.join(self.OutputPath,'%s' % PcdValueCommonName)) + 'INC = '
 else:
 MakeApp = MakeApp + PcdGccMakefile
-MakeApp = MakeApp + 'APPFILE = %s/%s\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = %s/%s.o 
%s.o\n' % (self.OutputPath, PcdValueInitName, 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon"))) + \
+PcdValueCommonPath = 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon.c"))
+MakeApp = MakeApp + 'all:%s\PcdValueCommon.c\n' % self.OutputPath
+MakeApp = MakeApp + '%s/PcdValueCommon.c : %s\n' % 
(self.OutputPath, PcdValueCommonPath)
+MakeApp = MakeApp + '\tcp -f %s %s/PcdValueCommon.c\n' % 
(PcdValueCommonPath, self.OutputPath)
+MakeApp = MakeApp + 'APPFILE = %s/%s\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = %s/%s.o 
%s.o\n' % (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,'%s' 
% PcdValueCommonName)) + \
   'include $(MAKEROOT)/Makefiles/app.makefile\n' + 
'INCLUDE +='
 
 IncSearchList = []
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#55744): https://edk2.groups.io/g/devel/message/55744
Mute This Topic: https://groups.io/mt/71874103/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH 1/1] BaseTools:copy the common PcdValueCommon.obj to output directory

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2568

PcdValueInit shares the same Edk2\BaseTools\Source\C\PcdValueCommon.obj.
To avoid the conflict, it should copy this file to its output directory,
If so, PcdValueCommon.obj file will be private for PcdValueInit

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Workspace/DscBuildData.py | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py 
b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 476c7edaf9da..15c4a0c4ab01 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -55,6 +55,7 @@ def _IsFieldValueAnArray (Value):
 return False
 
 PcdValueInitName = 'PcdValueInit'
+PcdValueCommonName = 'PcdValueCommon'
 
 PcdMainCHeader = '''
 /**
@@ -2634,10 +2635,14 @@ class DscBuildData(PlatformBuildClassObject):
 
 MakeApp = PcdMakefileHeader
 if sys.platform == "win32":
-MakeApp = MakeApp + 'APPFILE = %s\%s.exe\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = 
%s\%s.obj %s.obj\n' % (self.OutputPath, PcdValueInitName, 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon"))) + 'INC = '
+PcdValueCommonPath = 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon"))
+shutil.copy2('%s.obj' % PcdValueCommonPath, 
os.path.join(self.OutputPath,'%s.obj' % PcdValueCommonName))
+MakeApp = MakeApp + 'APPFILE = %s\%s.exe\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = 
%s\%s.obj %s.obj\n' % (self.OutputPath, PcdValueInitName, 
os.path.join(self.OutputPath,'%s' % PcdValueCommonName)) + 'INC = '
 else:
 MakeApp = MakeApp + PcdGccMakefile
-MakeApp = MakeApp + 'APPFILE = %s/%s\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = %s/%s.o 
%s.o\n' % (self.OutputPath, PcdValueInitName, 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon"))) + \
+PcdValueCommonPath = 
os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], 
"Source/C/Common/PcdValueCommon"))
+shutil.copy2('%s.o' % PcdValueCommonPath, 
os.path.join(self.OutputPath, '%s.o' % PcdValueCommonName))
+MakeApp = MakeApp + 'APPFILE = %s/%s\n' % (self.OutputPath, 
PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = %s/%s.o 
%s.o\n' % (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,'%s' 
% PcdValueCommonName)) + \
   'include $(MAKEROOT)/Makefiles/app.makefile\n' + 
'INCLUDE +='
 
 IncSearchList = []
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#55677): https://edk2.groups.io/g/devel/message/55677
Mute This Topic: https://groups.io/mt/71830146/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH V2] BaseTools:GuidedSectionTools.txt is not generated correctly

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538

For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.

*_*_*_LZMA_PATH = LzmaCompressPlatform

This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.

Signed-off-by: Zhiju.Fan 
Cc: Liming Gao 
Cc: Bob Feng 
---
Changed an issue with an incorrect full path in GuidedSectionTools

 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py | 4 +++-
 BaseTools/Source/Python/build/build.py | 3 ++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py 
b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
index d32178b00c93..478a5116fd7a 100644
--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
@@ -866,7 +866,9 @@ class PlatformAutoGen(AutoGen):
 Value += " " + 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
 else:
 Value = 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
-
+Def = '_'.join([self.BuildTarget, self.ToolChain, 
self.Arch, Tool, Attr])
+if 
self.Workspace.ToolDef.ToolsDefTxtDictionary.get(Def):
+
self.Workspace.ToolDef.ToolsDefTxtDictionary[Def] = Value
 if Attr == "PATH":
 # Don't put MAKE definition in the file
 if Tool != "MAKE":
diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index d841fefdc502..01c4c6fe2b84 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -2367,7 +2367,8 @@ class Build():
 # PATH environment variable.
 for dirInPath in os.environ['PATH'].split(os.pathsep):
 foundPath = os.path.join(dirInPath, tool)
-if os.path.exists(foundPath):
+BaseName, Ext = os.path.splitext(foundPath)
+if os.path.isdir(foundPath) or Ext:
 return os.path.realpath(foundPath)
 
 # If the tool was not found in the path then we just return
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#55571): https://edk2.groups.io/g/devel/message/55571
Mute This Topic: https://groups.io/mt/71766931/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH] BaseTools:GuidedSectionTools.txt is not generated correctly

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538

For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.

*_*_*_LZMA_PATH = LzmaCompressPlatform

This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.

Signed-off-by: Zhiju.Fan 
Cc: Liming Gao 
Cc: Bob Feng 
---
 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py 
b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
index d32178b00c..478a5116fd 100644
--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
@@ -866,7 +866,9 @@ class PlatformAutoGen(AutoGen):
 Value += " " + 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
 else:
 Value = 
self._BuildOptionWithToolDef(RetVal)[Tool][Attr]
-
+Def = '_'.join([self.BuildTarget, self.ToolChain, 
self.Arch, Tool, Attr])
+if 
self.Workspace.ToolDef.ToolsDefTxtDictionary.get(Def):
+
self.Workspace.ToolDef.ToolsDefTxtDictionary[Def] = Value
 if Attr == "PATH":
 # Don't put MAKE definition in the file
 if Tool != "MAKE":
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#55464): https://edk2.groups.io/g/devel/message/55464
Mute This Topic: https://groups.io/mt/71739757/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [Patch 1/1] BaseTools:build failure in CLANGPDB tool chain

[Fan, ZhijuX]

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2520

Incremental build failure in CLANGPDB tool chain on Windows host
The build failure is like below when do incremental build.
The root cause is in generated deps_target file. It has one line ":".

Signed-off-by: Zhiju.Fan 

Cc: Liming Gao 
Cc: Bob Feng 
---
 BaseTools/Source/Python/AutoGen/IncludesAutoGen.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py 
b/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
index ca9e02d19b4a..0a6314266f45 100644
--- a/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
@@ -111,6 +111,8 @@ class IncludesAutoGen():
 continue
 dependency_file = item.strip(" \\\n")
 dependency_file = dependency_file.strip('''"''')
+if dependency_file == '':
+continue
 if os.path.normpath(dependency_file +".deps") == abspath:
 continue
 filename = os.path.basename(dependency_file).strip()
-- 
2.18.0.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#54277): https://edk2.groups.io/g/devel/message/54277
Mute This Topic: https://groups.io/mt/71208485/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [Patch] BaseTools: Fixed build failure when using python38

[Fan, ZhijuX]

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2304

SyntaxWarning: "is" with a literal. Did you mean "=="?
Using "is" instead of "==" is an irregular syntax

Signed-off-by: Zhiju.Fan 
Cc: Bob C Feng 
Cc: Liming Gao 
---
 BaseTools/Source/Python/build/BuildReport.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/build/BuildReport.py 
b/BaseTools/Source/Python/build/BuildReport.py
index 880459d367..8efa869162 100644
--- a/BaseTools/Source/Python/build/BuildReport.py
+++ b/BaseTools/Source/Python/build/BuildReport.py
@@ -2042,7 +2042,7 @@ class FdReport(object):
 self.VPDBaseAddress = 0
 self.VPDSize = 0
 for index, FdRegion in enumerate(Fd.RegionList):
-if str(FdRegion.RegionType) is 'FILE' and Wa.Platform.VpdToolGuid 
in str(FdRegion.RegionDataList):
+if str(FdRegion.RegionType) == 'FILE' and Wa.Platform.VpdToolGuid 
in str(FdRegion.RegionDataList):
 self.VPDBaseAddress = self.FdRegionList[index].BaseAddress
 self.VPDSize = self.FdRegionList[index].Size
 break
-- 
2.18.0.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#54263): https://edk2.groups.io/g/devel/message/54263
Mute This Topic: https://groups.io/mt/71206432/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH V2] BaseTools:ECC fails to detect function header comments issue

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1523

When the keyword after @param doesn't match the actual function
parameter name, ECC doesn't detect such issue

The patch is going to fix this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---

Optimize the code to Give the CheckParamName function
an explicit return value

 BaseTools/Source/Python/Ecc/c.py | 18 +-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/Ecc/c.py b/BaseTools/Source/Python/Ecc/c.py
index a30122a45f..d21ea2b7a3 100644
--- a/BaseTools/Source/Python/Ecc/c.py
+++ b/BaseTools/Source/Python/Ecc/c.py
@@ -2554,6 +2554,22 @@ def CheckGeneralDoxygenCommentLayout(Str, StartLine, 
ErrorMsgList, CommentId= -1
 ErrorMsgList.append('Line %d : @retval appear before @param ' % 
StartLine)
 PrintErrorMsg(ERROR_DOXYGEN_CHECK_FUNCTION_HEADER, 'in Comment, 
@retval appear before @param  ', TableName, CommentId)
 
+def CheckParamName(ParamName, Tag):
+ParamList = Tag.split()
+if len(ParamList) > 1:
+ParamList[0] = ParamList[0].replace('@param', '')
+for Item in ParamList:
+if Item.find('[') > 0 and Item.find(']') > 0:
+continue
+if ParamName != Item.strip():
+return True
+else:
+return False
+else:
+return True
+else:
+return True
+
 def CheckFunctionHeaderConsistentWithDoxygenComment(FuncModifier, FuncHeader, 
FuncStartLine, CommentStr, CommentStartLine, ErrorMsgList, CommentId= -1, 
TableName=''):
 
 ParamList = GetParamList(FuncHeader)
@@ -2608,7 +2624,7 @@ def 
CheckFunctionHeaderConsistentWithDoxygenComment(FuncModifier, FuncHeader, Fu
 
PrintErrorMsg(ERROR_DOXYGEN_CHECK_FUNCTION_HEADER, 'in Comment, <%s> does NOT 
have %s ' % ((TagPartList[0] + ' ' + TagPartList[1]).replace('\n', 
'').replace('\r', ''), '[' + InOutStr + ']'), TableName, CommentId)
 
 
-if Tag.find(ParamName) == -1 and ParamName != 'VOID' and ParamName 
!= 'void':
+if (Tag.find(ParamName) == -1 or CheckParamName(ParamName, Tag)) 
and ParamName != 'VOID' and ParamName != 'void':
 ErrorMsgList.append('Line %d : in Comment, <%s> does NOT 
consistent with parameter name %s ' % (CommentStartLine, (TagPartList[0] + ' ' 
+ TagPartList[1]).replace('\n', '').replace('\r', ''), ParamName))
 PrintErrorMsg(ERROR_DOXYGEN_CHECK_FUNCTION_HEADER, 'in 
Comment, <%s> does NOT consistent with parameter name %s ' % ((TagPartList[0] + 
' ' + TagPartList[1]).replace('\n', '').replace('\r', ''), ParamName), 
TableName, CommentId)
 Index += 1
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#53424): https://edk2.groups.io/g/devel/message/53424
Mute This Topic: https://groups.io/mt/69952600/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:ECC fails to detect function header comments issue

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1523

When the keyword after @param doesn't match the actual function
parameter name, ECC doesn't detect such issue

The patch is going to fix this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Ecc/c.py | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/Ecc/c.py b/BaseTools/Source/Python/Ecc/c.py
index a30122a45f..e42463952d 100644
--- a/BaseTools/Source/Python/Ecc/c.py
+++ b/BaseTools/Source/Python/Ecc/c.py
@@ -2554,6 +2554,20 @@ def CheckGeneralDoxygenCommentLayout(Str, StartLine, 
ErrorMsgList, CommentId= -1
 ErrorMsgList.append('Line %d : @retval appear before @param ' % 
StartLine)
 PrintErrorMsg(ERROR_DOXYGEN_CHECK_FUNCTION_HEADER, 'in Comment, 
@retval appear before @param  ', TableName, CommentId)
 
+def CheckParamName(ParamName, Tag):
+ParamList = Tag.split()
+if len(ParamList) > 1:
+ParamList.pop(0)
+for Item in ParamList:
+if Item.find('[') > 0 and Item.find(']') > 0:
+continue
+if ParamName != Item.strip():
+return True
+else:
+return False
+else:
+return True
+
 def CheckFunctionHeaderConsistentWithDoxygenComment(FuncModifier, FuncHeader, 
FuncStartLine, CommentStr, CommentStartLine, ErrorMsgList, CommentId= -1, 
TableName=''):
 
 ParamList = GetParamList(FuncHeader)
@@ -2608,7 +2622,7 @@ def 
CheckFunctionHeaderConsistentWithDoxygenComment(FuncModifier, FuncHeader, Fu
 
PrintErrorMsg(ERROR_DOXYGEN_CHECK_FUNCTION_HEADER, 'in Comment, <%s> does NOT 
have %s ' % ((TagPartList[0] + ' ' + TagPartList[1]).replace('\n', 
'').replace('\r', ''), '[' + InOutStr + ']'), TableName, CommentId)
 
 
-if Tag.find(ParamName) == -1 and ParamName != 'VOID' and ParamName 
!= 'void':
+if (Tag.find(ParamName) == -1 or CheckParamName(ParamName, Tag)) 
and ParamName != 'VOID' and ParamName != 'void':
 ErrorMsgList.append('Line %d : in Comment, <%s> does NOT 
consistent with parameter name %s ' % (CommentStartLine, (TagPartList[0] + ' ' 
+ TagPartList[1]).replace('\n', '').replace('\r', ''), ParamName))
 PrintErrorMsg(ERROR_DOXYGEN_CHECK_FUNCTION_HEADER, 'in 
Comment, <%s> does NOT consistent with parameter name %s ' % ((TagPartList[0] + 
' ' + TagPartList[1]).replace('\n', '').replace('\r', ''), ParamName), 
TableName, CommentId)
 Index += 1
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#53395): https://edk2.groups.io/g/devel/message/53395
Mute This Topic: https://groups.io/mt/69928321/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH V2] BaseTools:fix Ecc tool issue for check StructPcd

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2142

gAdvancedFeaturePkgTokenSpaceGuid.PcdSmbiosType0BiosInformation|
{0x0}|SMBIOS_TABLE_TYPE0|0x8001 {

  IndustryStandard/SmBios.h

  MdePkg/MdePkg.dec
  AdvancedFeaturePkg/AdvancedFeaturePkg.dec
  }

If there's a PcdStructHF or PcdStructPKGs in StructPcd,
EccTool report error,IndexError: list index out of range

This patch is going to fix this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---

V2:Optimize the code to Reduce patch redundancy and
update the copyright to 2020

 .../Python/Ecc/MetaFileWorkspace/MetaFileParser.py | 71 +-
 1 file changed, 69 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py 
b/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
index 1576565455..9c27c8e16a 100644
--- a/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
+++ b/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
@@ -1,7 +1,7 @@
 ## @file
 # This file is used to parse meta files
 #
-# Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.
+# Copyright (c) 2008 - 2020, Intel Corporation. All rights reserved.
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 #
 
@@ -13,7 +13,7 @@ import Common.LongFilePathOs as os
 import re
 import time
 import copy
-
+from hashlib import md5
 import Common.EdkLogger as EdkLogger
 import Common.GlobalData as GlobalData
 import Ecc.EccGlobalData as EccGlobalData
@@ -1498,6 +1498,10 @@ class DecParser(MetaFileParser):
 self.TblFile = EccGlobalData.gDb.TblFile
 self.FileID = -1
 
+self._CurrentStructurePcdName = ""
+self._include_flag = False
+self._package_flag = False
+
 ## Parser starter
 def Start(self):
 Content = ''
@@ -1692,6 +1696,62 @@ class DecParser(MetaFileParser):
 File=self.MetaFile, Line=self._LineIndex+1)
 self._ValueList[0] = ''
 
+def ParsePcdName(self,namelist):
+if "[" in namelist[1]:
+pcdname = namelist[1][:namelist[1].index("[")]
+arrayindex = namelist[1][namelist[1].index("["):]
+namelist[1] = pcdname
+if len(namelist) == 2:
+namelist.append(arrayindex)
+else:
+namelist[2] = ".".join((arrayindex,namelist[2]))
+return namelist
+
+def StructPcdParser(self):
+self._ValueList[0] = self._CurrentStructurePcdName
+
+if "|" not in self._CurrentLine:
+if "" == self._CurrentLine:
+self._include_flag = True
+self._package_flag = False
+self._ValueList = None
+return
+if "" == self._CurrentLine:
+self._package_flag = True
+self._ValueList = None
+self._include_flag = False
+return
+
+if self._include_flag:
+self._ValueList[1] = "_" + 
md5(self._CurrentLine.encode('utf-8')).hexdigest()
+self._ValueList[2] = self._CurrentLine
+if self._package_flag and "}" != self._CurrentLine:
+self._ValueList[1] = "_" + 
md5(self._CurrentLine.encode('utf-8')).hexdigest()
+self._ValueList[2] = self._CurrentLine
+if self._CurrentLine == "}":
+self._package_flag = False
+self._include_flag = False
+self._ValueList = None
+else:
+PcdTockens = self._CurrentLine.split(TAB_VALUE_SPLIT)
+PcdNames = self.ParsePcdName(PcdTockens[0].split(TAB_SPLIT))
+if len(PcdNames) == 2:
+if PcdNames[1].strip().endswith("]"):
+PcdName = PcdNames[1][:PcdNames[1].index('[')]
+Index = PcdNames[1][PcdNames[1].index('['):]
+self._ValueList[0] = TAB_SPLIT.join((PcdNames[0], PcdName))
+self._ValueList[1] = Index
+self._ValueList[2] = PcdTockens[1]
+else:
+self._CurrentStructurePcdName = ""
+else:
+if self._CurrentStructurePcdName != 
TAB_SPLIT.join(PcdNames[:2]):
+EdkLogger.error('Parser', FORMAT_INVALID, "Pcd Name does 
not match: %s and %s " % (
+self._CurrentStructurePcdName, 
TAB_SPLIT.join(PcdNames[:2])),
+File=self.MetaFile, Line=self._LineIndex + 
1)
+self._ValueList[1] = TAB_SPLIT.join(PcdNames[2:])
+self._ValueList[2] = PcdTockens[1]
+
 ## PCD sections parser
 #
 #   [PcdsFixedAtBuild]
@@ -1702,6 +1762,9 @@ class DecParser(MetaFileParser):
 #
 @ParseMacro
 def _PcdParser(self):
+if self._CurrentStructurePcdName:
+self.StructPcdParser()
+return
 TokenList = GetSplitValueList(self._CurrentLine, 

[edk2-devel] [PATCH V2] BaseTools:fix Ecc tool issue for check StructPcd

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2142

gAdvancedFeaturePkgTokenSpaceGuid.PcdSmbiosType0BiosInformation|
{0x0}|SMBIOS_TABLE_TYPE0|0x8001 {

  IndustryStandard/SmBios.h

  MdePkg/MdePkg.dec
  AdvancedFeaturePkg/AdvancedFeaturePkg.dec
  }

If there's a PcdStructHF or PcdStructPKGs in StructPcd,
EccTool report error,IndexError: list index out of range

This patch is going to fix this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 .../Python/Ecc/MetaFileWorkspace/MetaFileParser.py | 71 +-
 1 file changed, 69 insertions(+), 2 deletions(-)

Optimize the code to Reduce patch redundancy and update the copyright to 2020

diff --git a/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py 
b/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
index 1576565455..9c27c8e16a 100644
--- a/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
+++ b/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
@@ -1,7 +1,7 @@
 ## @file
 # This file is used to parse meta files
 #
-# Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.
+# Copyright (c) 2008 - 2020, Intel Corporation. All rights reserved.
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 #
 
@@ -13,7 +13,7 @@ import Common.LongFilePathOs as os
 import re
 import time
 import copy
-
+from hashlib import md5
 import Common.EdkLogger as EdkLogger
 import Common.GlobalData as GlobalData
 import Ecc.EccGlobalData as EccGlobalData
@@ -1498,6 +1498,10 @@ class DecParser(MetaFileParser):
 self.TblFile = EccGlobalData.gDb.TblFile
 self.FileID = -1
 
+self._CurrentStructurePcdName = ""
+self._include_flag = False
+self._package_flag = False
+
 ## Parser starter
 def Start(self):
 Content = ''
@@ -1692,6 +1696,62 @@ class DecParser(MetaFileParser):
 File=self.MetaFile, Line=self._LineIndex+1)
 self._ValueList[0] = ''
 
+def ParsePcdName(self,namelist):
+if "[" in namelist[1]:
+pcdname = namelist[1][:namelist[1].index("[")]
+arrayindex = namelist[1][namelist[1].index("["):]
+namelist[1] = pcdname
+if len(namelist) == 2:
+namelist.append(arrayindex)
+else:
+namelist[2] = ".".join((arrayindex,namelist[2]))
+return namelist
+
+def StructPcdParser(self):
+self._ValueList[0] = self._CurrentStructurePcdName
+
+if "|" not in self._CurrentLine:
+if "" == self._CurrentLine:
+self._include_flag = True
+self._package_flag = False
+self._ValueList = None
+return
+if "" == self._CurrentLine:
+self._package_flag = True
+self._ValueList = None
+self._include_flag = False
+return
+
+if self._include_flag:
+self._ValueList[1] = "_" + 
md5(self._CurrentLine.encode('utf-8')).hexdigest()
+self._ValueList[2] = self._CurrentLine
+if self._package_flag and "}" != self._CurrentLine:
+self._ValueList[1] = "_" + 
md5(self._CurrentLine.encode('utf-8')).hexdigest()
+self._ValueList[2] = self._CurrentLine
+if self._CurrentLine == "}":
+self._package_flag = False
+self._include_flag = False
+self._ValueList = None
+else:
+PcdTockens = self._CurrentLine.split(TAB_VALUE_SPLIT)
+PcdNames = self.ParsePcdName(PcdTockens[0].split(TAB_SPLIT))
+if len(PcdNames) == 2:
+if PcdNames[1].strip().endswith("]"):
+PcdName = PcdNames[1][:PcdNames[1].index('[')]
+Index = PcdNames[1][PcdNames[1].index('['):]
+self._ValueList[0] = TAB_SPLIT.join((PcdNames[0], PcdName))
+self._ValueList[1] = Index
+self._ValueList[2] = PcdTockens[1]
+else:
+self._CurrentStructurePcdName = ""
+else:
+if self._CurrentStructurePcdName != 
TAB_SPLIT.join(PcdNames[:2]):
+EdkLogger.error('Parser', FORMAT_INVALID, "Pcd Name does 
not match: %s and %s " % (
+self._CurrentStructurePcdName, 
TAB_SPLIT.join(PcdNames[:2])),
+File=self.MetaFile, Line=self._LineIndex + 
1)
+self._ValueList[1] = TAB_SPLIT.join(PcdNames[2:])
+self._ValueList[2] = PcdTockens[1]
+
 ## PCD sections parser
 #
 #   [PcdsFixedAtBuild]
@@ -1702,6 +1762,9 @@ class DecParser(MetaFileParser):
 #
 @ParseMacro
 def _PcdParser(self):
+if self._CurrentStructurePcdName:
+self.StructPcdParser()
+return
 TokenList = GetSplitValueList(self._CurrentLine, 

[edk2-devel] [PATCH V3] BaseTools:Change the case rules for ECC check pointer names

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2087

In CryptHkdf.c  line 42

  EVP_PKEY_CTX *pHkdfCtx;

Variable pHkdfCtx begins with lower case 'p',
which should be acceptable because it it is a pointer.
(Refer to CCS_2_1_Draft, 4.3.3.3)

So ECC tool should be improved to handle issues like this.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Ecc/Check.py | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

 update the copyright to 2020

diff --git a/BaseTools/Source/Python/Ecc/Check.py 
b/BaseTools/Source/Python/Ecc/Check.py
index b68cecddfd..0fdc7e35c1 100644
--- a/BaseTools/Source/Python/Ecc/Check.py
+++ b/BaseTools/Source/Python/Ecc/Check.py
@@ -1,7 +1,7 @@
 ## @file
 # This file is used to define checkpoints used by ECC tool
 #
-# Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.
+# Copyright (c) 2008 - 2020, Intel Corporation. All rights reserved.
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 #
 from __future__ import absolute_import
@@ -1469,13 +1469,14 @@ class Check(object):
 EdkLogger.quiet("Checking naming convention of variable name ...")
 Pattern = re.compile(r'^[A-Zgm]+\S*[a-z]\S*$')
 
-SqlCommand = """select ID, Name from %s where Model = %s""" % 
(FileTable, MODEL_IDENTIFIER_VARIABLE)
+SqlCommand = """select ID, Name, Modifier from %s where Model = 
%s""" % (FileTable, MODEL_IDENTIFIER_VARIABLE)
 RecordSet = EccGlobalData.gDb.TblFile.Exec(SqlCommand)
 for Record in RecordSet:
 Var = Record[1]
+Modifier = Record[2]
 if Var.startswith('CONST'):
 Var = Var[5:].lstrip()
-if not Pattern.match(Var):
+if not Pattern.match(Var) and not (Modifier.endswith('*') and 
Var.startswith('p')):
 if not 
EccGlobalData.gException.IsException(ERROR_NAMING_CONVENTION_CHECK_VARIABLE_NAME,
 Record[1]):
 
EccGlobalData.gDb.TblReport.Insert(ERROR_NAMING_CONVENTION_CHECK_VARIABLE_NAME, 
OtherMsg="The variable name [%s] does not follow the rules" % (Record[1]), 
BelongsToTable=FileTable, BelongsToItem=Record[0])
 
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#53145): https://edk2.groups.io/g/devel/message/53145
Mute This Topic: https://groups.io/mt/69596952/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH V2] BaseTools:Fix GenFds issue for BuildOption replace GenFdsOption

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2455

BuildOption is used by TargetTxtClassObj.py
GenFdsOption is used by GenFds.py
When the GenFds tool is used alone (e.g. python3 -m GenFds.GenFds -h)
With the OptionParser function, the first detected function
prints the help message

import TargetTxtClassObj to GenFds,
The BuildOption will be executed and replace GenFdsOption

We removed all objects associated with this problem that
were created directly during the import process
(e.g. BuildOption, BuildTarget = MyOptionParser(),
 TargetTxt = TargetTxtDict())

The Patch is going to fix this issue

Signed-off-by: Zhiju.Fan 
Cc: Liming Gao 
Cc: Bob Feng 
---
 BaseTools/Source/Python/AutoGen/BuildEngine.py |  56 +++
 BaseTools/Source/Python/Common/GlobalData.py   |   2 +-
 .../Source/Python/Common/TargetTxtClassObject.py   |  64 +--
 .../Source/Python/Common/ToolDefClassObject.py |  48 +++---
 BaseTools/Source/Python/Common/buildoptions.py |  93 --
 BaseTools/Source/Python/GenFds/GenFds.py   |   4 +-
 .../Source/Python/GenFds/GenFdsGlobalVariable.py   |  17 ++--
 BaseTools/Source/Python/Workspace/DscBuildData.py  |   9 +-
 BaseTools/Source/Python/build/build.py |  32 +--
 BaseTools/Source/Python/build/buildoptions.py  | 105 +
 10 files changed, 261 insertions(+), 169 deletions(-)
 delete mode 100644 BaseTools/Source/Python/Common/buildoptions.py
 create mode 100644 BaseTools/Source/Python/build/buildoptions.py

The method by which the optimization function gets the 
parameter value
add bugzilla to patch

diff --git a/BaseTools/Source/Python/AutoGen/BuildEngine.py 
b/BaseTools/Source/Python/AutoGen/BuildEngine.py
index 069a3a1c9d..d602414ca4 100644
--- a/BaseTools/Source/Python/AutoGen/BuildEngine.py
+++ b/BaseTools/Source/Python/AutoGen/BuildEngine.py
@@ -20,7 +20,7 @@ from Common.BuildToolError import *
 from Common.Misc import tdict, PathClass
 from Common.StringUtils import NormPath
 from Common.DataType import *
-from Common.TargetTxtClassObject import TargetTxt
+from Common.TargetTxtClassObject import TargetTxtDict
 gDefaultBuildRuleFile = 'build_rule.txt'
 AutoGenReqBuildRuleVerNum = '0.1'
 
@@ -588,24 +588,42 @@ class BuildRule:
 _UnknownSection: SkipSection,
 }
 
-def GetBuildRule():
-BuildRuleFile = None
-if TAB_TAT_DEFINES_BUILD_RULE_CONF in TargetTxt.TargetTxtDictionary:
-BuildRuleFile = 
TargetTxt.TargetTxtDictionary[TAB_TAT_DEFINES_BUILD_RULE_CONF]
-if not BuildRuleFile:
-BuildRuleFile = gDefaultBuildRuleFile
-RetVal = BuildRule(BuildRuleFile)
-if RetVal._FileVersion == "":
-RetVal._FileVersion = AutoGenReqBuildRuleVerNum
-else:
-if RetVal._FileVersion < AutoGenReqBuildRuleVerNum :
-# If Build Rule's version is less than the version number required 
by the tools, halting the build.
-EdkLogger.error("build", AUTOGEN_ERROR,
-ExtraData="The version number [%s] of 
build_rule.txt is less than the version number required by the AutoGen.(the 
minimum required version number is [%s])"\
- % (RetVal._FileVersion, 
AutoGenReqBuildRuleVerNum))
-return RetVal
-
-BuildRuleObj = GetBuildRule()
+class ToolBuildRule():
+
+def __new__(cls, *args, **kw):
+if not hasattr(cls, '_instance'):
+orig = super(ToolBuildRule, cls)
+cls._instance = orig.__new__(cls, *args, **kw)
+return cls._instance
+
+def __init__(self):
+if not hasattr(self, 'ToolBuildRule'):
+self._ToolBuildRule = None
+
+@property
+def ToolBuildRule(self):
+if not self._ToolBuildRule:
+self._GetBuildRule()
+return self._ToolBuildRule
+
+def _GetBuildRule(self):
+BuildRuleFile = None
+TargetObj = TargetTxtDict()
+TargetTxt = TargetObj.Target
+if TAB_TAT_DEFINES_BUILD_RULE_CONF in TargetTxt.TargetTxtDictionary:
+BuildRuleFile = 
TargetTxt.TargetTxtDictionary[TAB_TAT_DEFINES_BUILD_RULE_CONF]
+if not BuildRuleFile:
+BuildRuleFile = gDefaultBuildRuleFile
+RetVal = BuildRule(BuildRuleFile)
+if RetVal._FileVersion == "":
+RetVal._FileVersion = AutoGenReqBuildRuleVerNum
+else:
+if RetVal._FileVersion < AutoGenReqBuildRuleVerNum :
+# If Build Rule's version is less than the version number 
required by the tools, halting the build.
+EdkLogger.error("build", AUTOGEN_ERROR,
+ExtraData="The version number [%s] of 
build_rule.txt is less than the version number required by the AutoGen.(the 
minimum required version number is [%s])"\
+ % (RetVal._FileVersion, 
AutoGenReqBuildRuleVerNum))
+self._ToolBuildRule = RetVal
 
 # This acts like the main() function for the script, unless it is 

[edk2-devel] [PATCH V2] BaseTools:Change the case rules for ECC check pointer names

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2087

In CryptHkdf.c  line 42

  EVP_PKEY_CTX *pHkdfCtx;

Variable pHkdfCtx begins with lower case 'p',
which should be acceptable because it it is a pointer.
(Refer to CCS_2_1_Draft, 4.3.3.3)

So ECC tool should be improved to handle issues like this.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Ecc/Check.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/Ecc/Check.py 
b/BaseTools/Source/Python/Ecc/Check.py
index 2180818609..f927d8e4d3 100644
--- a/BaseTools/Source/Python/Ecc/Check.py
+++ b/BaseTools/Source/Python/Ecc/Check.py
@@ -1469,13 +1469,14 @@ class Check(object):
 EdkLogger.quiet("Checking naming convention of variable name ...")
 Pattern = re.compile(r'^[A-Zgm]+\S*[a-z]\S*$')
 
-SqlCommand = """select ID, Name from %s where Model = %s""" % 
(FileTable, MODEL_IDENTIFIER_VARIABLE)
+SqlCommand = """select ID, Name, Modifier from %s where Model = 
%s""" % (FileTable, MODEL_IDENTIFIER_VARIABLE)
 RecordSet = EccGlobalData.gDb.TblFile.Exec(SqlCommand)
 for Record in RecordSet:
 Var = Record[1]
+Modifier = Record[2]
 if Var.startswith('CONST'):
 Var = Var[5:].lstrip()
-if not Pattern.match(Var):
+if not Pattern.match(Var) and not (Modifier.endswith('*') and 
Var.startswith('p')):
 if not 
EccGlobalData.gException.IsException(ERROR_NAMING_CONVENTION_CHECK_VARIABLE_NAME,
 Record[1]):
 
EccGlobalData.gDb.TblReport.Insert(ERROR_NAMING_CONVENTION_CHECK_VARIABLE_NAME, 
OtherMsg="The variable name [%s] does not follow the rules" % (Record[1]), 
BelongsToTable=FileTable, BelongsToItem=Record[0])
 
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#53120): https://edk2.groups.io/g/devel/message/53120
Mute This Topic: https://groups.io/mt/69594869/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:fix Ecc tool issue for check StructPcd

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2142

gAdvancedFeaturePkgTokenSpaceGuid.PcdSmbiosType0BiosInformation|
{0x0}|SMBIOS_TABLE_TYPE0|0x8001 {

  IndustryStandard/SmBios.h

  MdePkg/MdePkg.dec
  AdvancedFeaturePkg/AdvancedFeaturePkg.dec
  }

If there's a PcdStructHF or PcdStructPKGs in StructPcd,
EccTool report error,IndexError: list index out of range

This patch is going to fix this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 .../Python/Ecc/MetaFileWorkspace/MetaFileParser.py | 362 +
 1 file changed, 220 insertions(+), 142 deletions(-)

diff --git a/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py 
b/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
index 1576565455..f7b12d8855 100644
--- a/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
+++ b/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
@@ -13,7 +13,7 @@ import Common.LongFilePathOs as os
 import re
 import time
 import copy
-
+from hashlib import md5
 import Common.EdkLogger as EdkLogger
 import Common.GlobalData as GlobalData
 import Ecc.EccGlobalData as EccGlobalData
@@ -1498,6 +1498,10 @@ class DecParser(MetaFileParser):
 self.TblFile = EccGlobalData.gDb.TblFile
 self.FileID = -1
 
+self._CurrentStructurePcdName = ""
+self._include_flag = False
+self._package_flag = False
+
 ## Parser starter
 def Start(self):
 Content = ''
@@ -1692,6 +1696,17 @@ class DecParser(MetaFileParser):
 File=self.MetaFile, Line=self._LineIndex+1)
 self._ValueList[0] = ''
 
+def ParsePcdName(self,namelist):
+if "[" in namelist[1]:
+pcdname = namelist[1][:namelist[1].index("[")]
+arrayindex = namelist[1][namelist[1].index("["):]
+namelist[1] = pcdname
+if len(namelist) == 2:
+namelist.append(arrayindex)
+else:
+namelist[2] = ".".join((arrayindex,namelist[2]))
+return namelist
+
 ## PCD sections parser
 #
 #   [PcdsFixedAtBuild]
@@ -1702,153 +1717,216 @@ class DecParser(MetaFileParser):
 #
 @ParseMacro
 def _PcdParser(self):
-TokenList = GetSplitValueList(self._CurrentLine, TAB_VALUE_SPLIT, 1)
-self._ValueList[0:1] = GetSplitValueList(TokenList[0], TAB_SPLIT)
-# check PCD information
-if self._ValueList[0] == '' or self._ValueList[1] == '':
-EdkLogger.error('Parser', FORMAT_INVALID, "No token space GUID or 
PCD name specified",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-File=self.MetaFile, Line=self._LineIndex+1)
-# check PCD datum information
-if len(TokenList) < 2 or TokenList[1] == '':
-EdkLogger.error('Parser', FORMAT_INVALID, "No PCD Datum 
information given",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-File=self.MetaFile, Line=self._LineIndex+1)
-
-
-ValueRe  = re.compile(r'^\s*L?\".*\|.*\"')
-PtrValue = ValueRe.findall(TokenList[1])
-
-# Has VOID* type string, may contain "|" character in the string.
-if len(PtrValue) != 0:
-ptrValueList = re.sub(ValueRe, '', TokenList[1])
-ValueList= GetSplitValueList(ptrValueList)
-ValueList[0] = PtrValue[0]
-else:
-ValueList = GetSplitValueList(TokenList[1])
-
-
-# check if there's enough datum information given
-if len(ValueList) != 3:
-EdkLogger.error('Parser', FORMAT_INVALID, "Invalid PCD Datum 
information given",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-File=self.MetaFile, Line=self._LineIndex+1)
-# check default value
-if ValueList[0] == '':
-EdkLogger.error('Parser', FORMAT_INVALID, "Missing DefaultValue in 
PCD Datum information",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-File=self.MetaFile, Line=self._LineIndex+1)
-# check datum type
-if ValueList[1] == '':
-EdkLogger.error('Parser', FORMAT_INVALID, "Missing DatumType in 
PCD Datum information",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-File=self.MetaFile, Line=self._LineIndex+1)
-# check token of the PCD
-if ValueList[2] == '':
-EdkLogger.error('Parser', FORMAT_INVALID, "Missing Token in PCD 
Datum information",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-   

[edk2-devel] [PATCH] BaseTools:Change the case rules for ECC check pointer names

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2087

In CryptHkdf.c  line 42

  EVP_PKEY_CTX *pHkdfCtx;

Variable pHkdfCtx begins with lower case 'p',
which should be acceptable because it it is a pointer.
(Refer to CCS_2_1_Draft, 4.3.3.3)

So ECC tool should be improved to handle issues like this.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Ecc/Check.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/Ecc/Check.py 
b/BaseTools/Source/Python/Ecc/Check.py
index 2180818609..f927d8e4d3 100644
--- a/BaseTools/Source/Python/Ecc/Check.py
+++ b/BaseTools/Source/Python/Ecc/Check.py
@@ -1469,13 +1469,14 @@ class Check(object):
 EdkLogger.quiet("Checking naming convention of variable name ...")
 Pattern = re.compile(r'^[A-Zgm]+\S*[a-z]\S*$')
 
-SqlCommand = """select ID, Name from %s where Model = %s""" % 
(FileTable, MODEL_IDENTIFIER_VARIABLE)
+SqlCommand = """select ID, Name, Modifier from %s where Model = 
%s""" % (FileTable, MODEL_IDENTIFIER_VARIABLE)
 RecordSet = EccGlobalData.gDb.TblFile.Exec(SqlCommand)
 for Record in RecordSet:
 Var = Record[1]
+Modifier = Record[2]
 if Var.startswith('CONST'):
 Var = Var[5:].lstrip()
-if not Pattern.match(Var):
+if not Pattern.match(Var) and not Modifier.endswith('*'):
 if not 
EccGlobalData.gException.IsException(ERROR_NAMING_CONVENTION_CHECK_VARIABLE_NAME,
 Record[1]):
 
EccGlobalData.gDb.TblReport.Insert(ERROR_NAMING_CONVENTION_CHECK_VARIABLE_NAME, 
OtherMsg="The variable name [%s] does not follow the rules" % (Record[1]), 
BelongsToTable=FileTable, BelongsToItem=Record[0])
 
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#52969): https://edk2.groups.io/g/devel/message/52969
Mute This Topic: https://groups.io/mt/69498799/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:Reconfig reset environment value set by edksetup script

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1985

Reconfig option should not only update build config files. This option
should reset environment value set by edksetup script, such as
EDK_TOOLS_PATH/PYTHONPATH. If so, this option will be valuable for
the user to switch the different WORKSPACEs.

We can locate BASETOOLS by setting the WORKSPACE in a directory
separate from BASETOOLS,but it can't switch workspaces.
for example, set WORKSPACE=C:\work\edk2 in directory C:\
run C:\work\edk2\edksetup in directory C:\
switch workspaces, set WORKSPACE=C:\workspace\edk2
run C:\workspace\edk2\edksetup in directory C:\
This situation does not apply with Reconfig,
So I use the "clean" option to clear the environment variables
associated with edksetup
run "C:\workspace\edk2\edksetup clean" in directory C:\,
run "C:\workspace\edk2\edksetup" in directory C:\

This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 edksetup.bat | 28 ++--
 edksetup.sh  | 31 ---
 2 files changed, 54 insertions(+), 5 deletions(-)

diff --git a/edksetup.bat b/edksetup.bat
index 024f57a4b7..97b55659ac 100755
--- a/edksetup.bat
+++ b/edksetup.bat
@@ -25,6 +25,29 @@
 pushd .
 cd %~dp0
 
+if /I "%1" NEQ "Reconfig" (
+  if /I "%1" NEQ "EnvClean" (
+goto SetEnv
+)
+  )
+
+set EDK_TOOLS_BIN=
+set CONF_PATH=
+set BASETOOLS_PYTHON_SOURCE=
+set EDK_TOOLS_PATH=
+set WORKSPACE=
+
+if /I "%1"=="Reconfig" (
+  echo reset environment value set by edksetup script
+  goto SetEnv
+)
+
+if /I "%1"=="EnvClean" (
+  echo clean environment value set by edksetup script
+  goto end
+)
+
+:SetEnv
 if not defined WORKSPACE (
   goto SetWorkSpace
 )
@@ -142,9 +165,10 @@ if "%1"=="" goto end
 
 :Usage
   @echo.
-  @echo  Usage: "%0 [-h | -help | --help | /h | /help | /?] [Reconfig] 
[Rebuild] [ForceRebuild] [VS2019] [VS2017] [VS2015] [VS2013] [VS2012]"
+  @echo  Usage: "%0 [-h | -help | --help | /h | /help | /?] [Reconfig] 
[Rebuild] [EnvClean] [ForceRebuild] [VS2019] [VS2017] [VS2015] [VS2013] 
[VS2012]"
   @echo.
-  @echo Reconfig   Reinstall target.txt, tools_def.txt and 
build_rule.txt.
+  @echo Reconfig   Reinstall target.txt, tools_def.txt， 
build_rule.txt and reset environment value set by edksetup script.
+  @echo EnvClean   Clean environment value by edksetup script.
   @echo RebuildPerform incremental rebuild of BaseTools 
binaries.
   @echo ForceRebuild   Force a full rebuild of BaseTools binaries.
   @echo VS2012 Set the env for VS2012 build.
diff --git a/edksetup.sh b/edksetup.sh
index 06d2f041e6..10ce6fb752 100755
--- a/edksetup.sh
+++ b/edksetup.sh
@@ -33,6 +33,8 @@ function HelpMsg()
   echo "  --reconfigOverwrite the WORKSPACE/Conf/*.txt files with 
the"
   echo "template files from the BaseTools/Conf 
directory."
   echo
+  echo "  --envcleanClean environment value by edksetup script."
+  echo
   echo Please note: This script must be \'sourced\' so the environment can be 
changed.
   echo ". $SCRIPTNAME"
   echo "source $SCRIPTNAME"
@@ -44,6 +46,20 @@ function SetWorkspace()
   # If WORKSPACE is already set, then we can return right now
   #
   export PYTHONHASHSEED=1
+  if [ -n $RECONFIG ]
+  then
+echo reset environment value set by edksetup script
+WORKSPACE=
+CONF_PATH=
+  fi
+
+  if [ -n $ENVCLEAN ]
+  then
+echo clean environment value by edksetup script
+WORKSPACE=
+CONF_PATH=
+return 0
+  fi
   if [ -n "$WORKSPACE" ]
   then
 return 0
@@ -177,9 +193,14 @@ function SetupPython()
 
 function SourceEnv()
 {
-  SetWorkspace &&
-  SetupEnv
-  SetupPython
+  if [ -n $ENVCLEAN ]
+  then
+SetWorkspace
+  else
+SetWorkspace &&
+SetupEnv
+SetupPython
+  fi
 }
 
 I=$#
@@ -194,6 +215,10 @@ do
   RECONFIG=TRUE
   shift
 ;;
+--envclean)
+  ENVCLEAN=TRUE
+  shift
+;;
 *)
   HelpMsg
   break
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#52576): https://edk2.groups.io/g/devel/message/52576
Mute This Topic: https://groups.io/mt/69268972/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:replaces the two offending quotes by ascii quotes

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2423

That commit 13c5e34a1b introduces the first two UTF-8
characters (the quote ') in an otherwise all-ascii file.

In Conf\tools_def.template
There is tow lines of
  Notes: Since this tool chain is obsolete, it doesn't enable
  the compiler option for included header file list generation,

we replaces the two offending quotes by proper ascii quotes
The patch is going to fix this issue

Signed-off-by: Zhiju.Fan 
Cc: Liming Gao 
Cc: Bob Feng 
---
 BaseTools/Conf/tools_def.template | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Conf/tools_def.template 
b/BaseTools/Conf/tools_def.template
index 41a6ca246e..feee2bbf16 100755
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -337,7 +337,7 @@ DEFINE DTC_BIN = ENV(DTC_PREFIX)dtc
 # Required to build platforms or ACPI tables:
 #   Microsoft ASL ACPI Compiler (asl.exe) v4.0.0 
from
 #   
http://download.microsoft.com/download/2/c/1/2c16c7e0-96c1-40f5-81fc-3e4bf7b65496/microsoft_asl_compiler-v4-0-0.msi
-#Notes: Since this tool chain is obsolete, it 
doesn’t enable the compiler option for included header file list generation,
+#Notes: Since this tool chain is obsolete, it doesn't 
enable the compiler option for included header file list generation,
 #   and lose the incremental build capability.
 #   RVCTLINUX   -unix-   Requires:
 # ARM C/C++ Compiler, 5.00
@@ -345,7 +345,7 @@ DEFINE DTC_BIN = ENV(DTC_PREFIX)dtc
 # Required to build platforms or ACPI tables:
 #   Intel(r) ACPI Compiler from
 #   https://acpica.org/downloads
-#Notes: Since this tool chain is obsolete, it 
doesn’t enable the compiler option for included header file list generation,
+#Notes: Since this tool chain is obsolete, it doesn't 
enable the compiler option for included header file list generation,
 #   and lose the incremental build capability.
 # * Commented out - All versions of VS2005 use the same standard install 
directory
 #
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#52280): https://edk2.groups.io/g/devel/message/52280
Mute This Topic: https://groups.io/mt/68753868/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:Enhance the way to handling included dsc file

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2400

In Dsc Parser, included dsc file is parsed always no matter
if its condition is False

  gUefiOvmfPkgTokenSpaceGuid.test1|FALSE
!if gUefiOvmfPkgTokenSpaceGuid.test1 == FALSE
  !include OvmfPkg/test1.dsc
!else
  !include OvmfPkg/test2.dsc
!endif

In the above case, since the conditional result is FALSE,
"test2.dsc" is not parsed.
The patch avoids processing redundant dsc files and improves
the way Tool handles them.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 .../Source/Python/Workspace/MetaFileParser.py  | 79 +++---
 1 file changed, 40 insertions(+), 39 deletions(-)

diff --git a/BaseTools/Source/Python/Workspace/MetaFileParser.py 
b/BaseTools/Source/Python/Workspace/MetaFileParser.py
index 8a665b118e..a3b6edbd15 100644
--- a/BaseTools/Source/Python/Workspace/MetaFileParser.py
+++ b/BaseTools/Source/Python/Workspace/MetaFileParser.py
@@ -1612,46 +1612,47 @@ class DscParser(MetaFileParser):
 # First search the include file under the same directory as DSC 
file
 #
 IncludedFile1 = PathClass(IncludedFile, self.MetaFile.Dir)
-ErrorCode, ErrorInfo1 = IncludedFile1.Validate()
-if ErrorCode != 0:
-#
-# Also search file under the WORKSPACE directory
-#
-IncludedFile1 = PathClass(IncludedFile, GlobalData.gWorkspace)
-ErrorCode, ErrorInfo2 = IncludedFile1.Validate()
+if self._Enabled:
+ErrorCode, ErrorInfo1 = IncludedFile1.Validate()
 if ErrorCode != 0:
-EdkLogger.error('parser', ErrorCode, 
File=self._FileWithError,
-Line=self._LineIndex + 1, 
ExtraData=ErrorInfo1 + "\n" + ErrorInfo2)
-
-self._FileWithError = IncludedFile1
-
-FromItem = self._Content[self._ContentIndex - 1][0]
-if self._InSubsection:
-Owner = self._Content[self._ContentIndex - 1][8]
-else:
-Owner = self._Content[self._ContentIndex - 1][0]
-IncludedFileTable = MetaFileStorage(self._RawTable.DB, 
IncludedFile1, MODEL_FILE_DSC, False, FromItem=FromItem)
-Parser = DscParser(IncludedFile1, self._FileType, self._Arch, 
IncludedFileTable,
-   Owner=Owner, From=FromItem)
-
-self.IncludedFiles.add (IncludedFile1)
-
-# set the parser status with current status
-Parser._SectionName = self._SectionName
-Parser._SubsectionType = self._SubsectionType
-Parser._InSubsection = self._InSubsection
-Parser._SectionType = self._SectionType
-Parser._Scope = self._Scope
-Parser._Enabled = self._Enabled
-# Parse the included file
-Parser.StartParse()
-# Insert all records in the table for the included file into dsc 
file table
-Records = IncludedFileTable.GetAll()
-if Records:
-self._Content[self._ContentIndex:self._ContentIndex] = Records
-self._Content.pop(self._ContentIndex - 1)
-self._ValueList = None
-self._ContentIndex -= 1
+#
+# Also search file under the WORKSPACE directory
+#
+IncludedFile1 = PathClass(IncludedFile, 
GlobalData.gWorkspace)
+ErrorCode, ErrorInfo2 = IncludedFile1.Validate()
+if ErrorCode != 0:
+EdkLogger.error('parser', ErrorCode, 
File=self._FileWithError,
+Line=self._LineIndex + 1, 
ExtraData=ErrorInfo1 + "\n" + ErrorInfo2)
+
+self._FileWithError = IncludedFile1
+
+FromItem = self._Content[self._ContentIndex - 1][0]
+if self._InSubsection:
+Owner = self._Content[self._ContentIndex - 1][8]
+else:
+Owner = self._Content[self._ContentIndex - 1][0]
+IncludedFileTable = MetaFileStorage(self._RawTable.DB, 
IncludedFile1, MODEL_FILE_DSC, False, FromItem=FromItem)
+Parser = DscParser(IncludedFile1, self._FileType, self._Arch, 
IncludedFileTable,
+   Owner=Owner, From=FromItem)
+
+self.IncludedFiles.add (IncludedFile1)
+
+# set the parser status with current status
+Parser._SectionName = self._SectionName
+Parser._SubsectionType = self._SubsectionType
+Parser._InSubsection = self._InSubsection
+Parser._SectionType = self._SectionType
+Parser._Scope = self._Scope
+Parser._Enabled = self._Enabled
+# Parse the included file
+Parser.StartParse()
+# Insert all records 

[edk2-devel] [PATCH V2] BaseTools:fix issue for decode the stdout/stderr byte arrays

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2365

This patch is to fix a build tool regression issue which was introduced
by commit 8ddec24dea74.

compiler output message includes localized string.
So build failed when code decode the stdout/stderr byte arrays.
The cause of the build failed is that Commit 8ddec24dea74
removed "errors='ignore'".

The build tool does not need to deal with localized string,
so we need to add "errors='ignore'".

this function is only invoked for structure PCDs.
Build failed if structurePcd is used in platform dsc file.
The patch is going to fixed this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Workspace/DscBuildData.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py 
b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 9192077f90..901d95a413 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -1752,7 +1752,7 @@ class DscBuildData(PlatformBuildClassObject):
 except:
 EdkLogger.error('Build', COMMAND_FAILURE, 'Can not execute 
command: %s' % Command)
 Result = Process.communicate()
-return Process.returncode, Result[0].decode(), Result[1].decode()
+return Process.returncode, Result[0].decode(errors='ignore'), 
Result[1].decode(errors='ignore')
 
 @staticmethod
 def IntToCString(Value, ValueSize):
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#51515): https://edk2.groups.io/g/devel/message/51515
Mute This Topic: https://groups.io/mt/64522458/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:Change judgment symbol "is" to "==" for python3.8

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2304

build.py and BuildReport.py warning using python 3.8
SyntaxWarning: "is" with a literal. Did you mean "=="?

For comparison of two strings use "==" instead of "is"
The patch is going to fixed this issue.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/BuildReport.py | 2 +-
 BaseTools/Source/Python/build/build.py   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/build/BuildReport.py 
b/BaseTools/Source/Python/build/BuildReport.py
index 880459d367..8efa869162 100644
--- a/BaseTools/Source/Python/build/BuildReport.py
+++ b/BaseTools/Source/Python/build/BuildReport.py
@@ -2042,7 +2042,7 @@ class FdReport(object):
 self.VPDBaseAddress = 0
 self.VPDSize = 0
 for index, FdRegion in enumerate(Fd.RegionList):
-if str(FdRegion.RegionType) is 'FILE' and Wa.Platform.VpdToolGuid 
in str(FdRegion.RegionDataList):
+if str(FdRegion.RegionType) == 'FILE' and Wa.Platform.VpdToolGuid 
in str(FdRegion.RegionDataList):
 self.VPDBaseAddress = self.FdRegionList[index].BaseAddress
 self.VPDSize = self.FdRegionList[index].Size
 break
diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index 07f1f21b5d..4b31356a42 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -2064,7 +2064,7 @@ class Build():
 if Fdf.CurrentFdName and Fdf.CurrentFdName in Fdf.Profile.FdDict:
 FdDict = Fdf.Profile.FdDict[Fdf.CurrentFdName]
 for FdRegion in FdDict.RegionList:
-if str(FdRegion.RegionType) is 'FILE' and 
self.Platform.VpdToolGuid in str(FdRegion.RegionDataList):
+if str(FdRegion.RegionType) == 'FILE' and 
self.Platform.VpdToolGuid in str(FdRegion.RegionDataList):
 if int(FdRegion.Offset) % 8 != 0:
 EdkLogger.error("build", FORMAT_INVALID, 'The VPD 
Base Address %s must be 8-byte aligned.' % (FdRegion.Offset))
 Wa.FdfProfile = Fdf.Profile
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#51484): https://edk2.groups.io/g/devel/message/51484
Mute This Topic: https://groups.io/mt/64318144/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH V3] BaseTools:fix regression issue for platform .map file

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2363

This patch is to fix a build tool regression issue which was introduced
by commit b8ac0b7f28.This issue caused map file lost the line of IMAGE=***.
For example,in Ovmf.map, there is no line of (IMAGE= ) under
each of modules item.

The path to the efi file generated by each module is written on this line
The purpose of this line is add the debug image full path.
there is no information about the module in the map file other than FVName,
it allows us to quickly know which module this part corresponds to.

In commit b8ac0b7f28,add a line ("self.BuildModules = []") in function,
but it's used to calculate the variable ModuleList in the following code.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/build.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index bcd832c525..07f1f21b5d 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -2267,6 +2267,10 @@ class Build():
 self.CreateAsBuiltInf()
 if GlobalData.gBinCacheDest:
 self.UpdateBuildCache()
+#
+# Get Module List
+#
+ModuleList = {ma.Guid.upper(): ma for ma in self.BuildModules}
 self.BuildModules = []
 self.MakeTime += int(round((time.time() - MakeContiue)))
 #
@@ -2285,10 +2289,6 @@ class Build():
 #
 if (Arch == 'IA32' or Arch == 'ARM') and 
self.LoadFixAddress != 0x and self.LoadFixAddress >= 
0x1:
 EdkLogger.error("build", PARAMETER_INVALID, 
"FIX_LOAD_TOP_MEMORY_ADDRESS can't be set to larger than or equal to 4G for the 
platorm with IA32 or ARM arch modules")
-#
-# Get Module List
-#
-ModuleList = {ma.Guid.upper():ma for ma in 
self.BuildModules}
 
 #
 # Rebase module to the preferred memory address before 
GenFds
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#51180): https://edk2.groups.io/g/devel/message/51180
Mute This Topic: https://groups.io/mt/61631498/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH V2] BaseTools:fix regression issue for platform .map file

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2363

The line of IMAGE=*** is missing in platform .map file.For example,
in Ovmf.map, there is no line of (IMAGE= ) under each of modules item.
This is a regression issue.

this patch is going to fix this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/build.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index bcd832c525..07f1f21b5d 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -2267,6 +2267,10 @@ class Build():
 self.CreateAsBuiltInf()
 if GlobalData.gBinCacheDest:
 self.UpdateBuildCache()
+#
+# Get Module List
+#
+ModuleList = {ma.Guid.upper(): ma for ma in self.BuildModules}
 self.BuildModules = []
 self.MakeTime += int(round((time.time() - MakeContiue)))
 #
@@ -2285,10 +2289,6 @@ class Build():
 #
 if (Arch == 'IA32' or Arch == 'ARM') and 
self.LoadFixAddress != 0x and self.LoadFixAddress >= 
0x1:
 EdkLogger.error("build", PARAMETER_INVALID, 
"FIX_LOAD_TOP_MEMORY_ADDRESS can't be set to larger than or equal to 4G for the 
platorm with IA32 or ARM arch modules")
-#
-# Get Module List
-#
-ModuleList = {ma.Guid.upper():ma for ma in 
self.BuildModules}
 
 #
 # Rebase module to the preferred memory address before 
GenFds
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#50989): https://edk2.groups.io/g/devel/message/50989
Mute This Topic: https://groups.io/mt/60997993/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:fixed Build failed issue for Non-English OS

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2365

Build failed on Non-English OS if structurePcd is used in platform
dsc file.
When the output of some functions is converted to code, 
Because different OS Character encoding form differently,
there may be problems with some functions

The patch is going to fixed this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Workspace/DscBuildData.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py 
b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 9192077f90..901d95a413 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -1752,7 +1752,7 @@ class DscBuildData(PlatformBuildClassObject):
 except:
 EdkLogger.error('Build', COMMAND_FAILURE, 'Can not execute 
command: %s' % Command)
 Result = Process.communicate()
-return Process.returncode, Result[0].decode(), Result[1].decode()
+return Process.returncode, Result[0].decode(errors='ignore'), 
Result[1].decode(errors='ignore')
 
 @staticmethod
 def IntToCString(Value, ValueSize):
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#50936): https://edk2.groups.io/g/devel/message/50936
Mute This Topic: https://groups.io/mt/60828668/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:fix regression issue for platform .map file

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2363

The line of IMAGE=*** is missing in platform .map file.For example,
in Ovmf.map, there is no line of (IMAGE= ) under each of modules item.
This is a regression issue.

this patch is going to fix this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/build.py | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index bcd832c525..e24f040a93 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -2216,6 +2216,7 @@ class Build():
 Wa, self.BuildModules = 
self.PerformAutoGen(BuildTarget,ToolChain)
 Pa = Wa.AutoGenObjectList[0]
 GlobalData.gAutoGenPhase = False
+self.LoadFixAddress = Wa.Platform.LoadFixAddress
 
 if GlobalData.gBinCacheSource:
 EdkLogger.quiet("Total cache hit driver num: %s, cache 
miss driver num: %s" % (len(set(self.HashSkipModules)), 
len(set(self.BuildModules
@@ -2267,6 +2268,10 @@ class Build():
 self.CreateAsBuiltInf()
 if GlobalData.gBinCacheDest:
 self.UpdateBuildCache()
+#
+# Get Module List
+#
+ModuleList = {ma.Guid.upper(): ma for ma in self.BuildModules}
 self.BuildModules = []
 self.MakeTime += int(round((time.time() - MakeContiue)))
 #
@@ -2285,10 +2290,6 @@ class Build():
 #
 if (Arch == 'IA32' or Arch == 'ARM') and 
self.LoadFixAddress != 0x and self.LoadFixAddress >= 
0x1:
 EdkLogger.error("build", PARAMETER_INVALID, 
"FIX_LOAD_TOP_MEMORY_ADDRESS can't be set to larger than or equal to 4G for the 
platorm with IA32 or ARM arch modules")
-#
-# Get Module List
-#
-ModuleList = {ma.Guid.upper():ma for ma in 
self.BuildModules}
 
 #
 # Rebase module to the preferred memory address before 
GenFds
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#50934): https://edk2.groups.io/g/devel/message/50934
Mute This Topic: https://groups.io/mt/60786170/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

Re: [edk2-devel] [PATCH V2] BaseTools:Add [packages] section in dsc file

[Fan, ZhijuX]

Hi:

regression test:
pass build for minplatforms, WhitleyRp, TigerLake

functional test:

Define a PCD in a dec file (e.g. StandaloneMmPkg.dec) that none of the modules 
depend on,
it is defined in a [PcdsFixedAtBuild] or [PcdsFeatureFlag] section
Create the [Packages] section in the DSC file and add " StandaloneMmPkg.dec" in 
[Packages] section

Test whether this PCD can be used in conditional statements in DSC/FDF files


Any question, please let me know. Thanks.

Best Regards
Fan Zhiju



> -Original Message-
> From: Gao, Liming 
> Sent: Friday, November 15, 2019 9:55 AM
> To: Fan, ZhijuX ; devel@edk2.groups.io
> Cc: Feng, Bob C 
> Subject: RE: [PATCH V2] BaseTools:Add [packages] section in dsc file
> 
> Zhiju:
>   Can you show what test are done for this new support?
> 
> Thanks
> Liming
> >-Original Message-
> >From: Fan, ZhijuX
> >Sent: Thursday, November 14, 2019 9:28 AM
> >To: devel@edk2.groups.io
> >Cc: Gao, Liming ; Feng, Bob C
> >
> >Subject: [PATCH V2] BaseTools:Add [packages] section in dsc file
> >
> >BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2270
> >
> >Currently a PCD (e.g. FeaturePCD) cannot be used in a conditional
> >statement in a DSC/FDF file without a module in the build referencing
> >the PCD package DEC file.
> >
> >An example implementation that to support this is to allow a [Packages]
> >section in the DSC file to list additional package dependencies for PCD
> >references in the package DSC/FDF files.
> >
> >this patch is going to  add the ability to have the [packages] section
> >defined in the DSC file
> >
> >Cc: Liming Gao 
> >Cc: Bob Feng 
> >Signed-off-by: Zhiju.Fan 
> >---
> > BaseTools/Source/Python/AutoGen/ModuleAutoGen.py   | 29
> >+-
> > BaseTools/Source/Python/AutoGen/PlatformAutoGen.py |  1 +
> > .../Source/Python/AutoGen/WorkspaceAutoGen.py  |  1 +
> > BaseTools/Source/Python/Common/DataType.py |  1 +
> > BaseTools/Source/Python/Workspace/DscBuildData.py  | 23
> >-
> > .../Source/Python/Workspace/MetaFileParser.py  | 14 +++
> > .../Source/Python/Workspace/WorkspaceCommon.py |  2 ++
> > .../Source/Python/Workspace/WorkspaceDatabase.py   |  4 +++
> > 8 files changed, 68 insertions(+), 7 deletions(-)
> >
> >diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> >b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> >index f0812b6887..e6d6c43810 100755
> >--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> >+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> >@@ -462,14 +462,31 @@ class ModuleAutoGen(AutoGen):
> > def BuildCommand(self):
> > return self.PlatformInfo.BuildCommand
> >
> >-## Get object list of all packages the module and its dependent 
> >libraries
> >belong to
> >+## Get Module package and Platform package
> >+#
> >+#   @retval list The list of package object
> >+#
> >+@cached_property
> >+def PackageList(self):
> >+PkagList = []
> >+if self.Module.Packages:
> >+PkagList.extend(self.Module.Packages)
> >+Platform = self.BuildDatabase[self.PlatformInfo.MetaFile,
> >+ self.Arch,
> >self.BuildTarget, self.ToolChain]
> >+for Package in Platform.Packages:
> >+if Package in PkagList:
> >+continue
> >+PkagList.append(Package)
> >+return PkagList
> >+
> >+## Get object list of all packages the module and its dependent
> >+ libraries
> >belong to and the Platform depends on
> > #
> > #   @retval listThe list of package object
> > #
> > @cached_property
> > def DerivedPackageList(self):
> > PackageList = []
> >-for M in [self.Module] + self.DependentLibraryList:
> >+PackageList.extend(self.PackageList)
> >+for M in self.DependentLibraryList:
> > for Package in M.Packages:
> > if Package in PackageList:
> > continue
> >@@ -938,13 +955,13 @@ class ModuleAutoGen(AutoGen):
> > self.Targets
> > return self._FileTypes
> >
> >-## Get the list of package object the module depends on
> >+## Get the list of package object the module depends on and the
> >+ Platform
> >depends on
> > #
> > #   @retval listThe package object list
> > #
> > @cached_property
> > def DependentP

[edk2-devel] [PATCH V2] BaseTools:Add [packages] section in dsc file

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2270

Currently a PCD (e.g. FeaturePCD) cannot be used in a conditional
statement in a DSC/FDF file without a module in the build referencing
the PCD package DEC file.

An example implementation that to support this is to allow a [Packages]
section in the DSC file to list additional package dependencies for PCD
references in the package DSC/FDF files.

this patch is going to  add the ability to have the [packages] section
defined in the DSC file

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py   | 29 +-
 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py |  1 +
 .../Source/Python/AutoGen/WorkspaceAutoGen.py  |  1 +
 BaseTools/Source/Python/Common/DataType.py |  1 +
 BaseTools/Source/Python/Workspace/DscBuildData.py  | 23 -
 .../Source/Python/Workspace/MetaFileParser.py  | 14 +++
 .../Source/Python/Workspace/WorkspaceCommon.py |  2 ++
 .../Source/Python/Workspace/WorkspaceDatabase.py   |  4 +++
 8 files changed, 68 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py 
b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index f0812b6887..e6d6c43810 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -462,14 +462,31 @@ class ModuleAutoGen(AutoGen):
 def BuildCommand(self):
 return self.PlatformInfo.BuildCommand
 
-## Get object list of all packages the module and its dependent libraries 
belong to
+## Get Module package and Platform package
+#
+#   @retval list The list of package object
+#
+@cached_property
+def PackageList(self):
+PkagList = []
+if self.Module.Packages:
+PkagList.extend(self.Module.Packages)
+Platform = self.BuildDatabase[self.PlatformInfo.MetaFile, self.Arch, 
self.BuildTarget, self.ToolChain]
+for Package in Platform.Packages:
+if Package in PkagList:
+continue
+PkagList.append(Package)
+return PkagList
+
+## Get object list of all packages the module and its dependent libraries 
belong to and the Platform depends on
 #
 #   @retval listThe list of package object
 #
 @cached_property
 def DerivedPackageList(self):
 PackageList = []
-for M in [self.Module] + self.DependentLibraryList:
+PackageList.extend(self.PackageList)
+for M in self.DependentLibraryList:
 for Package in M.Packages:
 if Package in PackageList:
 continue
@@ -938,13 +955,13 @@ class ModuleAutoGen(AutoGen):
 self.Targets
 return self._FileTypes
 
-## Get the list of package object the module depends on
+## Get the list of package object the module depends on and the Platform 
depends on
 #
 #   @retval listThe package object list
 #
 @cached_property
 def DependentPackageList(self):
-return self.Module.Packages
+return self.PackageList
 
 ## Return the list of auto-generated code file
 #
@@ -1101,7 +1118,7 @@ class ModuleAutoGen(AutoGen):
 RetVal.append(self.MetaFile.Dir)
 RetVal.append(self.DebugDir)
 
-for Package in self.Module.Packages:
+for Package in self.PackageList:
 PackageDir = mws.join(self.WorkspaceDir, Package.MetaFile.Dir)
 if PackageDir not in RetVal:
 RetVal.append(PackageDir)
@@ -1125,7 +1142,7 @@ class ModuleAutoGen(AutoGen):
 @cached_property
 def PackageIncludePathList(self):
 IncludesList = []
-for Package in self.Module.Packages:
+for Package in self.PackageList:
 PackageDir = mws.join(self.WorkspaceDir, Package.MetaFile.Dir)
 IncludesList = Package.Includes
 if Package._PrivateIncludes:
diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py 
b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
index debeb46f58..4c3cdf82d5 100644
--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
@@ -975,6 +975,7 @@ class PlatformAutoGen(AutoGen):
 continue
 ModuleData = self.BuildDatabase[ModuleFile, self.Arch, 
self.BuildTarget, self.ToolChain]
 RetVal.update(ModuleData.Packages)
+RetVal.update(self.Platform.Packages)
 return list(RetVal)
 
 @cached_property
diff --git a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py 
b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py
index 9d8040905e..fde48b4b27 100644
--- a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py
@@ -420,6 +420,7 @@ class WorkspaceAutoGen(AutoGen):
 continue
 ModuleData = 

[edk2-devel] [PATCH] BaseTools:Add [packages] section in dsc file

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2270

Currently a PCD (e.g. FeaturePCD) cannot be used in a conditional
statement in a DSC/FDF file without a module in the build referencing
the PCD package DEC file.

An example implementation that to support this is to allow a [Packages]
section in the DSC file to list additional package dependencies for PCD
references in the package DSC/FDF files.

this patch is going to  add the ability to have the [packages] section
defined in the DSC file

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py   | 28 +-
 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py |  1 +
 .../Source/Python/AutoGen/WorkspaceAutoGen.py  |  1 +
 BaseTools/Source/Python/Common/DataType.py |  1 +
 BaseTools/Source/Python/Workspace/DscBuildData.py  | 23 +-
 .../Source/Python/Workspace/MetaFileParser.py  | 14 +++
 .../Source/Python/Workspace/WorkspaceCommon.py |  2 ++
 .../Source/Python/Workspace/WorkspaceDatabase.py   |  5 
 8 files changed, 68 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py 
b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index f0812b6887..112707edb1 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -462,13 +462,29 @@ class ModuleAutoGen(AutoGen):
 def BuildCommand(self):
 return self.PlatformInfo.BuildCommand
 
-## Get object list of all packages the module and its dependent libraries 
belong to
+## Get Module package and Platform package
+#
+#   @retval list The list of package object
+#
+@cached_property
+def PackageList(self):
+PkagList = []
+if self.Module.Packages:
+PkagList.extend(self.Module.Packages)
+Platform = self.BuildDatabase[self.PlatformInfo.MetaFile, self.Arch, 
self.BuildTarget, self.ToolChain]
+for Package in Platform.Packages:
+if Package in PkagList:
+continue
+PkagList.append(Package)
+return PkagList
+
+## Get object list of all packages the module and its dependent libraries 
belong to and the Platform depends on
 #
 #   @retval listThe list of package object
 #
 @cached_property
 def DerivedPackageList(self):
-PackageList = []
+PackageList = self.PackageList
 for M in [self.Module] + self.DependentLibraryList:
 for Package in M.Packages:
 if Package in PackageList:
@@ -938,13 +954,13 @@ class ModuleAutoGen(AutoGen):
 self.Targets
 return self._FileTypes
 
-## Get the list of package object the module depends on
+## Get the list of package object the module depends on and the Platform 
depends on
 #
 #   @retval listThe package object list
 #
 @cached_property
 def DependentPackageList(self):
-return self.Module.Packages
+return self.PackageList
 
 ## Return the list of auto-generated code file
 #
@@ -1101,7 +1117,7 @@ class ModuleAutoGen(AutoGen):
 RetVal.append(self.MetaFile.Dir)
 RetVal.append(self.DebugDir)
 
-for Package in self.Module.Packages:
+for Package in self.PackageList:
 PackageDir = mws.join(self.WorkspaceDir, Package.MetaFile.Dir)
 if PackageDir not in RetVal:
 RetVal.append(PackageDir)
@@ -1125,7 +1141,7 @@ class ModuleAutoGen(AutoGen):
 @cached_property
 def PackageIncludePathList(self):
 IncludesList = []
-for Package in self.Module.Packages:
+for Package in self.PackageList:
 PackageDir = mws.join(self.WorkspaceDir, Package.MetaFile.Dir)
 IncludesList = Package.Includes
 if Package._PrivateIncludes:
diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py 
b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
index debeb46f58..4c3cdf82d5 100644
--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
@@ -975,6 +975,7 @@ class PlatformAutoGen(AutoGen):
 continue
 ModuleData = self.BuildDatabase[ModuleFile, self.Arch, 
self.BuildTarget, self.ToolChain]
 RetVal.update(ModuleData.Packages)
+RetVal.update(self.Platform.Packages)
 return list(RetVal)
 
 @cached_property
diff --git a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py 
b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py
index 9d8040905e..fde48b4b27 100644
--- a/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py
@@ -420,6 +420,7 @@ class WorkspaceAutoGen(AutoGen):
 continue
 ModuleData = self.BuildDatabase[ModuleFile, Arch, 
self.BuildTarget, self.ToolChain]
 

[edk2-devel] [PATCH] BaseTools:Reconfig reset environment value set by edksetup script

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1985

Reconfig option should not only update build config files. This option
should reset environment value set by edksetup script, such as
EDK_TOOLS_PATH/PYTHONPATH. If so, this option will be valuable for
the user to switch the different WORKSPACEs.

We can locate BASETOOLS by setting the WORKSPACE in a directory
separate from BASETOOLS,but it can't switch workspaces.
for example, set WORKSPACE=C:\work\edk2 in directory C:\
run C:\work\edk2\edksetup in directory C:\
switch workspaces, set WORKSPACE=C:\workspace\edk2
run C:\workspace\edk2\edksetup in directory C:\
This situation does not apply with Reconfig,
So I use the "clean" option to clear the environment variables
associated with edksetup
run "C:\workspace\edk2\edksetup clean" in directory C:\,
run "C:\workspace\edk2\edksetup" in directory C:\

This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 edksetup.bat | 28 ++--
 edksetup.sh  | 31 ---
 2 files changed, 54 insertions(+), 5 deletions(-)

diff --git a/edksetup.bat b/edksetup.bat
index 024f57a4b7..00075ca48e 100755
--- a/edksetup.bat
+++ b/edksetup.bat
@@ -25,6 +25,29 @@
 pushd .
 cd %~dp0
 
+if /I "%1" neq "Reconfig" (
+  if /I "%1" neq "EnvClean" (
+goto CheckCD
+  )
+)
+
+set EDK_TOOLS_BIN=
+set CONF_PATH=
+set BASETOOLS_PYTHON_SOURCE=
+set EDK_TOOLS_PATH=
+set WORKSPACE=
+
+if /I "%1"=="Reconfig" (
+  echo reset environment value set by edksetup script
+  goto CheckCD
+)
+
+if /I "%1"=="EnvClean" (
+  echo clean environment value set by edksetup script
+  goto end
+)
+
+:CheckCD
 if not defined WORKSPACE (
   goto SetWorkSpace
 )
@@ -142,9 +165,10 @@ if "%1"=="" goto end
 
 :Usage
   @echo.
-  @echo  Usage: "%0 [-h | -help | --help | /h | /help | /?] [Reconfig] 
[Rebuild] [ForceRebuild] [VS2019] [VS2017] [VS2015] [VS2013] [VS2012]"
+  @echo  Usage: "%0 [-h | -help | --help | /h | /help | /?] [Reconfig] 
[Rebuild] [EnvClean] [ForceRebuild] [VS2019] [VS2017] [VS2015] [VS2013] 
[VS2012]"
   @echo.
-  @echo Reconfig   Reinstall target.txt, tools_def.txt and 
build_rule.txt.
+  @echo Reconfig   Reinstall target.txt, tools_def.txt， 
build_rule.txt and reset environment value set by edksetup script.
+  @echo EnvClean   Clean environment value by edksetup script.
   @echo RebuildPerform incremental rebuild of BaseTools 
binaries.
   @echo ForceRebuild   Force a full rebuild of BaseTools binaries.
   @echo VS2012 Set the env for VS2012 build.
diff --git a/edksetup.sh b/edksetup.sh
index 06d2f041e6..211cdc7230 100755
--- a/edksetup.sh
+++ b/edksetup.sh
@@ -33,6 +33,8 @@ function HelpMsg()
   echo "  --reconfigOverwrite the WORKSPACE/Conf/*.txt files with 
the"
   echo "template files from the BaseTools/Conf 
directory."
   echo
+  echo "  --envcleanClean environment value by edksetup script."
+  echo
   echo Please note: This script must be \'sourced\' so the environment can be 
changed.
   echo ". $SCRIPTNAME"
   echo "source $SCRIPTNAME"
@@ -44,6 +46,20 @@ function SetWorkspace()
   # If WORKSPACE is already set, then we can return right now
   #
   export PYTHONHASHSEED=1
+  if [ -n $RECONFIG ]
+  then
+echo reset environment value set by edksetup script
+WORKSPACE=
+CONF_PATH=
+  fi
+
+  if [ -n $ENVCLEAN ]
+  then
+echo clean environment value by edksetup script
+WORKSPACE=
+CONF_PATH=
+return 0
+  fi
   if [ -n "$WORKSPACE" ]
   then
 return 0
@@ -177,9 +193,14 @@ function SetupPython()
 
 function SourceEnv()
 {
-  SetWorkspace &&
-  SetupEnv
-  SetupPython
+  if [ -n $ENVCLEAN ]
+  then
+SetWorkspace
+  else
+SetWorkspace &&
+SetupEnv &&
+SetupPython
+  fi
 }
 
 I=$#
@@ -194,6 +215,10 @@ do
   RECONFIG=TRUE
   shift
 ;;
+--envclean)
+ENVCLEAN=TRUE
+shift
+;; 
 *)
   HelpMsg
   break
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#48268): https://edk2.groups.io/g/devel/message/48268
Mute This Topic: https://groups.io/mt/34331535/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:fix Ecc tool issue for check StructPcd

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2142

gAdvancedFeaturePkgTokenSpaceGuid.PcdSmbiosType0BiosInformation|
{0x0}|SMBIOS_TABLE_TYPE0|0x8001 {

  IndustryStandard/SmBios.h

  MdePkg/MdePkg.dec
  AdvancedFeaturePkg/AdvancedFeaturePkg.dec
  }

If there's a PcdStructHF or PcdStructPKGs in StructPcd,
EccTool report error,IndexError: list index out of range

This patch is going to fix this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 .../Python/Ecc/MetaFileWorkspace/MetaFileParser.py | 362 +
 1 file changed, 220 insertions(+), 142 deletions(-)

diff --git a/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py 
b/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
index 1576565455..f7b12d8855 100644
--- a/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
+++ b/BaseTools/Source/Python/Ecc/MetaFileWorkspace/MetaFileParser.py
@@ -13,7 +13,7 @@ import Common.LongFilePathOs as os
 import re
 import time
 import copy
-
+from hashlib import md5
 import Common.EdkLogger as EdkLogger
 import Common.GlobalData as GlobalData
 import Ecc.EccGlobalData as EccGlobalData
@@ -1498,6 +1498,10 @@ class DecParser(MetaFileParser):
 self.TblFile = EccGlobalData.gDb.TblFile
 self.FileID = -1
 
+self._CurrentStructurePcdName = ""
+self._include_flag = False
+self._package_flag = False
+
 ## Parser starter
 def Start(self):
 Content = ''
@@ -1692,6 +1696,17 @@ class DecParser(MetaFileParser):
 File=self.MetaFile, Line=self._LineIndex+1)
 self._ValueList[0] = ''
 
+def ParsePcdName(self,namelist):
+if "[" in namelist[1]:
+pcdname = namelist[1][:namelist[1].index("[")]
+arrayindex = namelist[1][namelist[1].index("["):]
+namelist[1] = pcdname
+if len(namelist) == 2:
+namelist.append(arrayindex)
+else:
+namelist[2] = ".".join((arrayindex,namelist[2]))
+return namelist
+
 ## PCD sections parser
 #
 #   [PcdsFixedAtBuild]
@@ -1702,153 +1717,216 @@ class DecParser(MetaFileParser):
 #
 @ParseMacro
 def _PcdParser(self):
-TokenList = GetSplitValueList(self._CurrentLine, TAB_VALUE_SPLIT, 1)
-self._ValueList[0:1] = GetSplitValueList(TokenList[0], TAB_SPLIT)
-# check PCD information
-if self._ValueList[0] == '' or self._ValueList[1] == '':
-EdkLogger.error('Parser', FORMAT_INVALID, "No token space GUID or 
PCD name specified",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-File=self.MetaFile, Line=self._LineIndex+1)
-# check PCD datum information
-if len(TokenList) < 2 or TokenList[1] == '':
-EdkLogger.error('Parser', FORMAT_INVALID, "No PCD Datum 
information given",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-File=self.MetaFile, Line=self._LineIndex+1)
-
-
-ValueRe  = re.compile(r'^\s*L?\".*\|.*\"')
-PtrValue = ValueRe.findall(TokenList[1])
-
-# Has VOID* type string, may contain "|" character in the string.
-if len(PtrValue) != 0:
-ptrValueList = re.sub(ValueRe, '', TokenList[1])
-ValueList= GetSplitValueList(ptrValueList)
-ValueList[0] = PtrValue[0]
-else:
-ValueList = GetSplitValueList(TokenList[1])
-
-
-# check if there's enough datum information given
-if len(ValueList) != 3:
-EdkLogger.error('Parser', FORMAT_INVALID, "Invalid PCD Datum 
information given",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-File=self.MetaFile, Line=self._LineIndex+1)
-# check default value
-if ValueList[0] == '':
-EdkLogger.error('Parser', FORMAT_INVALID, "Missing DefaultValue in 
PCD Datum information",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-File=self.MetaFile, Line=self._LineIndex+1)
-# check datum type
-if ValueList[1] == '':
-EdkLogger.error('Parser', FORMAT_INVALID, "Missing DatumType in 
PCD Datum information",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-File=self.MetaFile, Line=self._LineIndex+1)
-# check token of the PCD
-if ValueList[2] == '':
-EdkLogger.error('Parser', FORMAT_INVALID, "Missing Token in PCD 
Datum information",
-ExtraData=self._CurrentLine + \
-  " 
(.|||)",
-   

[edk2-devel] [PATCH V2] BaseTools:Fix the issue that build report failed

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2201

An error occurs using special VpdPcd that is not used in the Inf file

In dsc:
[PcdsDynamicExVpd.common.DEFAULT]
gBoardModuleTokenSpaceGuid.test1|*|{CODE({
  {0x0}  // terminator
})}

In dec:
[PcdsDynamicEx]
# Vpd GPIO table
  gBoardModuleTokenSpaceGuid.test1|{0}|GPIO_INIT_CONFIG[]|0x5018 {
  
Library/GpioLib.h
  
MdePkg/MdePkg.dec
}
ValueError: invalid literal for int() with base 0: '*'

This Patch is going to fix issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
Old title:Change the way that get some VpdPcd information 

 BaseTools/Source/Python/build/BuildReport.py | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/build/BuildReport.py 
b/BaseTools/Source/Python/build/BuildReport.py
index 6b26f1c3b0..880459d367 100644
--- a/BaseTools/Source/Python/build/BuildReport.py
+++ b/BaseTools/Source/Python/build/BuildReport.py
@@ -1419,10 +1419,19 @@ class PcdReport(object):
 FileWrite(File, '%*s' % (self.MaxLen + 4, 
SkuInfo.VpdOffset))
 VPDPcdItem = (Pcd.TokenSpaceGuidCName + '.' + 
PcdTokenCName, SkuIdName, SkuInfo.VpdOffset, Pcd.MaxDatumSize, 
SkuInfo.DefaultValue)
 if VPDPcdItem not in VPDPcdList:
-VPDPcdList.append(VPDPcdItem)
+PcdGuidList = 
self.UnusedPcds.get(Pcd.TokenSpaceGuidCName)
+if PcdGuidList:
+PcdList = PcdGuidList.get(Pcd.Type)
+if not PcdList:
+VPDPcdList.append(VPDPcdItem)
+for VpdPcd in PcdList:
+if PcdTokenCName == VpdPcd.TokenCName:
+break
+else:
+VPDPcdList.append(VPDPcdItem)
 if IsStructure:
 FiledOverrideFlag = False
-OverrideValues = Pcd.SkuOverrideValues[Sku]
+OverrideValues = Pcd.SkuOverrideValues.get(Sku)
 if OverrideValues:
 Keys = list(OverrideValues.keys())
 OverrideFieldStruct = self.OverrideFieldValue(Pcd, 
OverrideValues[Keys[0]])
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#47573): https://edk2.groups.io/g/devel/message/47573
Mute This Topic: https://groups.io/mt/34197653/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:"--exclude" don't apply if parameter ends with separator

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1944

FormatDosFiles.py Intel\ServerSiliconPkg --exclude Library\SimRegisters\

Its parameter "Library\SimRegisters\" ends with '\'
but I can't seem to get it to exclude the SimRegisters directory

This patch is going to fix this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Scripts/FormatDosFiles.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/BaseTools/Scripts/FormatDosFiles.py 
b/BaseTools/Scripts/FormatDosFiles.py
index 3150bef3c2..e119334ded 100644
--- a/BaseTools/Scripts/FormatDosFiles.py
+++ b/BaseTools/Scripts/FormatDosFiles.py
@@ -53,6 +53,7 @@ def FormatFilesInDir(DirPath, ExtList, Args):
 FileNames[:] = [f for f in FileNames if f not in Args.Exclude]
 Continue = False
 for Path in Args.Exclude:
+Path = Path.strip('\\').strip('/')
 if not os.path.isdir(Path) and not os.path.isfile(Path):
 Path = os.path.join(ExcludeDir, Path)
 if os.path.isdir(Path) and Path.endswith(DirPath):
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#47564): https://edk2.groups.io/g/devel/message/47564
Mute This Topic: https://groups.io/mt/34197338/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:Change the way that get some VpdPcd information

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2201

An error occurs using special VpdPcd that is not used in the Inf file

In dsc:
[PcdsDynamicExVpd.common.DEFAULT]
gBoardModuleTokenSpaceGuid.test1|*|{CODE({
  {0x0}  // terminator
})}

In dec:
[PcdsDynamicEx]
# Vpd GPIO table
  gBoardModuleTokenSpaceGuid.test1|{0}|GPIO_INIT_CONFIG[]|0x5018 {
  
Library/GpioLib.h
  
MdePkg/MdePkg.dec
}
ValueError: invalid literal for int() with base 0: '*'

This Patch is going to fix issue by Change the way that get
some VpdPcd information

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/BuildReport.py | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/build/BuildReport.py 
b/BaseTools/Source/Python/build/BuildReport.py
index 6b26f1c3b0..880459d367 100644
--- a/BaseTools/Source/Python/build/BuildReport.py
+++ b/BaseTools/Source/Python/build/BuildReport.py
@@ -1419,10 +1419,19 @@ class PcdReport(object):
 FileWrite(File, '%*s' % (self.MaxLen + 4, 
SkuInfo.VpdOffset))
 VPDPcdItem = (Pcd.TokenSpaceGuidCName + '.' + 
PcdTokenCName, SkuIdName, SkuInfo.VpdOffset, Pcd.MaxDatumSize, 
SkuInfo.DefaultValue)
 if VPDPcdItem not in VPDPcdList:
-VPDPcdList.append(VPDPcdItem)
+PcdGuidList = 
self.UnusedPcds.get(Pcd.TokenSpaceGuidCName)
+if PcdGuidList:
+PcdList = PcdGuidList.get(Pcd.Type)
+if not PcdList:
+VPDPcdList.append(VPDPcdItem)
+for VpdPcd in PcdList:
+if PcdTokenCName == VpdPcd.TokenCName:
+break
+else:
+VPDPcdList.append(VPDPcdItem)
 if IsStructure:
 FiledOverrideFlag = False
-OverrideValues = Pcd.SkuOverrideValues[Sku]
+OverrideValues = Pcd.SkuOverrideValues.get(Sku)
 if OverrideValues:
 Keys = list(OverrideValues.keys())
 OverrideFieldStruct = self.OverrideFieldValue(Pcd, 
OverrideValues[Keys[0]])
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#47474): https://edk2.groups.io/g/devel/message/47474
Mute This Topic: https://groups.io/mt/34186106/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:Replace PlatformInfo with PlatformAutoGen for Moudle

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2188

build -p MdeModulePkg\MdeModulePkg.dsc -a IA32 -m
MdeModulePkg\Universal\PCD\Pei\Pcd.inf

Error:
AttributeError: 'PlatformInfo' object has no attribute
'DynamicPcdList'

The DSC data object used to build a separate module today 
is PlatformInfo rather than PlatformAutoGen
'PlatformAutoGen' object has attribute 'DynamicPcdList'

This patch is going to fixed this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/build.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/BaseTools/Source/Python/build/build.py 
b/BaseTools/Source/Python/build/build.py
index e81d3d6486..e845c139c9 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -1860,6 +1860,9 @@ class Build():
 Ma = ModuleAutoGen(Wa, Module, BuildTarget, 
ToolChain, Arch, self.PlatformFile,Pa.DataPipe)
 if Ma is None:
 continue
+if Ma.PcdIsDriver:
+Ma.PlatformInfo = Pa
+Ma.Workspace = Wa
 MaList.append(Ma)
 
 if GlobalData.gBinCacheSource and self.Target in 
[None, "", "all"]:
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#47179): https://edk2.groups.io/g/devel/message/47179
Mute This Topic: https://groups.io/mt/34112948/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:change some incorrect parameter defaults

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1858

for Dict={},There are pitfalls in the way this default parameter is set
and Dict is not used in functions, other functions have these two cases,
I will change some incorrect parameter defaults

This patch is going to fix this issue

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/BuildEngine.py |  4 +++-
 BaseTools/Source/Python/AutoGen/GenMake.py |  6 --
 BaseTools/Source/Python/Common/RangeExpression.py  |  4 +++-
 BaseTools/Source/Python/Common/StringUtils.py  | 10 +++---
 BaseTools/Source/Python/GenFds/AprioriSection.py   |  4 +++-
 BaseTools/Source/Python/GenFds/CompressSection.py  |  4 +++-
 BaseTools/Source/Python/GenFds/DataSection.py  |  4 +++-
 BaseTools/Source/Python/GenFds/EfiSection.py   |  4 +++-
 BaseTools/Source/Python/GenFds/FfsFileStatement.py |  5 -
 BaseTools/Source/Python/GenFds/FfsInfStatement.py  |  5 +++--
 BaseTools/Source/Python/GenFds/Fv.py   |  4 +++-
 BaseTools/Source/Python/GenFds/FvImageSection.py   |  4 +++-
 BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py |  2 +-
 BaseTools/Source/Python/GenFds/GuidSection.py  |  4 +++-
 BaseTools/Source/Python/GenFds/OptRomFileStatement.py  |  5 -
 BaseTools/Source/Python/GenFds/Region.py   |  4 +++-
 BaseTools/Source/Python/GenFds/Section.py  |  2 +-
 BaseTools/Source/Python/GenFds/UiSection.py|  4 +++-
 BaseTools/Source/Python/GenFds/VerSection.py   |  4 +++-
 19 files changed, 60 insertions(+), 23 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/BuildEngine.py 
b/BaseTools/Source/Python/AutoGen/BuildEngine.py
index bb91534477..069a3a1c9d 100644
--- a/BaseTools/Source/Python/AutoGen/BuildEngine.py
+++ b/BaseTools/Source/Python/AutoGen/BuildEngine.py
@@ -176,7 +176,9 @@ class FileBuildRule:
 CommandString = "\n\t".join(self.CommandList)
 return "%s : %s\n\t%s" % (DestString, SourceString, CommandString)
 
-def Instantiate(self, Macros={}):
+def Instantiate(self, Macros = None):
+if Macros is None:
+Macros = {}
 NewRuleObject = copy.copy(self)
 NewRuleObject.BuildTargets = {}
 NewRuleObject.DestFileList = []
diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py 
b/BaseTools/Source/Python/AutoGen/GenMake.py
index 4f85a93055..3185ebe368 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -205,10 +205,12 @@ class BuildFile(object):
 def GetRemoveDirectoryCommand(self, DirList):
 return [self._RD_TEMPLATE_[self._FileType] % {'dir':Dir} for Dir in 
DirList]
 
-def PlaceMacro(self, Path, MacroDefinitions={}):
+def PlaceMacro(self, Path, MacroDefinitions=None):
 if Path.startswith("$("):
 return Path
 else:
+if MacroDefinitions is None:
+MacroDefinitions = {}
 PathLength = len(Path)
 for MacroName in MacroDefinitions:
 MacroValue = MacroDefinitions[MacroName]
@@ -1762,4 +1764,4 @@ def GetDependencyList(AutoGenObject, FileCache, File, 
ForceList, SearchPathList)
 
 # This acts like the main() function for the script, unless it is 'import'ed 
into another script.
 if __name__ == '__main__':
-pass
\ No newline at end of file
+pass
diff --git a/BaseTools/Source/Python/Common/RangeExpression.py 
b/BaseTools/Source/Python/Common/RangeExpression.py
index e9296e03f6..039d281467 100644
--- a/BaseTools/Source/Python/Common/RangeExpression.py
+++ b/BaseTools/Source/Python/Common/RangeExpression.py
@@ -342,7 +342,9 @@ class RangeExpression(BaseExpression):
 raise BadExpression(ERR_STRING_EXPR % Operator)
 
 
-def __init__(self, Expression, PcdDataType, SymbolTable = {}):
+def __init__(self, Expression, PcdDataType, SymbolTable = None):
+if SymbolTable is None:
+SymbolTable = {}
 super(RangeExpression, self).__init__(self, Expression, PcdDataType, 
SymbolTable)
 self._NoProcess = False
 if not isinstance(Expression, type('')):
diff --git a/BaseTools/Source/Python/Common/StringUtils.py 
b/BaseTools/Source/Python/Common/StringUtils.py
index 0febbc0034..73dafa797a 100644
--- a/BaseTools/Source/Python/Common/StringUtils.py
+++ b/BaseTools/Source/Python/Common/StringUtils.py
@@ -243,8 +243,10 @@ def SplitModuleType(Key):
 #
 # @retval NewList   A new string list whose macros are replaced
 #
-def ReplaceMacros(StringList, MacroDefinitions={}, SelfReplacement=False):
+def ReplaceMacros(StringList, MacroDefinitions=None, SelfReplacement=False):
 NewList = []
+if MacroDefinitions is None:
+MacroDefinitions = {}
 for String in StringList:
 if isinstance(String, type('')):
 NewList.append(ReplaceMacro(String, MacroDefinitions, 
SelfReplacement))

[edk2-devel] [PATCH] BaseTools:Change the case rules for ECC check pointer names

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2087

In CryptHkdf.c  line 42

  EVP_PKEY_CTX *pHkdfCtx;

Variable pHkdfCtx begins with lower case 'p',
which should be acceptable because it it is a pointer.
(Refer to CCS_2_1_Draft, 4.3.3.3)

So ECC tool should be improved to handle issues like this.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Ecc/Check.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/Ecc/Check.py 
b/BaseTools/Source/Python/Ecc/Check.py
index 2180818609..f927d8e4d3 100644
--- a/BaseTools/Source/Python/Ecc/Check.py
+++ b/BaseTools/Source/Python/Ecc/Check.py
@@ -1469,13 +1469,14 @@ class Check(object):
 EdkLogger.quiet("Checking naming convention of variable name ...")
 Pattern = re.compile(r'^[A-Zgm]+\S*[a-z]\S*$')
 
-SqlCommand = """select ID, Name from %s where Model = %s""" % 
(FileTable, MODEL_IDENTIFIER_VARIABLE)
+SqlCommand = """select ID, Name, Modifier from %s where Model = 
%s""" % (FileTable, MODEL_IDENTIFIER_VARIABLE)
 RecordSet = EccGlobalData.gDb.TblFile.Exec(SqlCommand)
 for Record in RecordSet:
 Var = Record[1]
+Modifier = Record[2]
 if Var.startswith('CONST'):
 Var = Var[5:].lstrip()
-if not Pattern.match(Var):
+if not Pattern.match(Var) and not Modifier.endswith('*'):
 if not 
EccGlobalData.gException.IsException(ERROR_NAMING_CONVENTION_CHECK_VARIABLE_NAME,
 Record[1]):
 
EccGlobalData.gDb.TblReport.Insert(ERROR_NAMING_CONVENTION_CHECK_VARIABLE_NAME, 
OtherMsg="The variable name [%s] does not follow the rules" % (Record[1]), 
BelongsToTable=FileTable, BelongsToItem=Record[0])
 
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#47118): https://edk2.groups.io/g/devel/message/47118
Mute This Topic: https://groups.io/mt/34101382/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:Ecc handle another copyright format

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2057

Ecc can not handle the copyright format like

(C) Copyright 2015-2016 Hewlett Packard Enterprise Development LP

This will cause Ecc to report wrong information.

This patch is going to handle this format

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Ecc/c.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/Ecc/c.py b/BaseTools/Source/Python/Ecc/c.py
index a99b40a701..a30122a45f 100644
--- a/BaseTools/Source/Python/Ecc/c.py
+++ b/BaseTools/Source/Python/Ecc/c.py
@@ -2388,7 +2388,7 @@ def CheckFileHeaderDoxygenComments(FullFileName):
 PrintErrorMsg(ERROR_HEADER_CHECK_FILE, 'File header 
comment content should start with two spaces at each line', FileTable, ID)
 
 CommentLine = CommentLine.strip()
-if CommentLine.startswith('Copyright'):
+if CommentLine.startswith('Copyright') or ('Copyright' in 
CommentLine and CommentLine.lower().startswith('(c)')):
 NoCopyrightFlag = False
 if CommentLine.find('All rights reserved') == -1:
 for Copyright in EccGlobalData.gConfig.Copyright:
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#46956): https://edk2.groups.io/g/devel/message/46956
Mute This Topic: https://groups.io/mt/33161998/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:ECC need to handle lower case 'static'

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1778

We are going to add keyword (lower case) 'static' for functions and global
variables which are not referenced outside their current C file.

However, the ECC tool only recognizes upper case 'STATIC' at this moment.
This will lead to issue reports for new codes that follow the above coding
style.

This patch is going to handle lower case 'static'

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Ecc/Check.py | 2 +-
 BaseTools/Source/Python/Ecc/Configuration.py | 2 +-
 BaseTools/Source/Python/Ecc/EccToolError.py  | 2 +-
 BaseTools/Source/Python/Ecc/c.py | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/BaseTools/Source/Python/Ecc/Check.py 
b/BaseTools/Source/Python/Ecc/Check.py
index 86bb8562ba..2180818609 100644
--- a/BaseTools/Source/Python/Ecc/Check.py
+++ b/BaseTools/Source/Python/Ecc/Check.py
@@ -441,7 +441,7 @@ class Check(object):
 self.DeclCheckUnionType()
 
 
-# Check whether no use of int, unsigned, char, void, static, long in any 
.c, .h or .asl files.
+# Check whether no use of int, unsigned, char, void, long in any .c, .h or 
.asl files.
 def DeclCheckNoUseCType(self):
 if EccGlobalData.gConfig.DeclarationDataTypeCheckNoUseCType == '1' or 
EccGlobalData.gConfig.DeclarationDataTypeCheckAll == '1' or 
EccGlobalData.gConfig.CheckAll == '1':
 EdkLogger.quiet("Checking Declaration No use C type ...")
diff --git a/BaseTools/Source/Python/Ecc/Configuration.py 
b/BaseTools/Source/Python/Ecc/Configuration.py
index 57ae932ed2..9d9feaca5e 100644
--- a/BaseTools/Source/Python/Ecc/Configuration.py
+++ b/BaseTools/Source/Python/Ecc/Configuration.py
@@ -256,7 +256,7 @@ class Configuration(object):
 ## Declarations and Data Types Checking
 self.DeclarationDataTypeCheckAll = 0
 
-# Check whether no use of int, unsigned, char, void, static, long in 
any .c, .h or .asl files.
+# Check whether no use of int, unsigned, char, void, long in any .c, 
.h or .asl files.
 self.DeclarationDataTypeCheckNoUseCType = 1
 # Check whether the modifiers IN, OUT, OPTIONAL, and UNALIGNED are 
used only to qualify arguments to a function and should not appear in a data 
type declaration
 self.DeclarationDataTypeCheckInOutModifier = 1
diff --git a/BaseTools/Source/Python/Ecc/EccToolError.py 
b/BaseTools/Source/Python/Ecc/EccToolError.py
index 4b1bef6a3e..0ff3b42674 100644
--- a/BaseTools/Source/Python/Ecc/EccToolError.py
+++ b/BaseTools/Source/Python/Ecc/EccToolError.py
@@ -149,7 +149,7 @@ gEccErrorMessage = {
 ERROR_INCLUDE_FILE_CHECK_NAME : "No permission for the include file with 
same names",
 
 ERROR_DECLARATION_DATA_TYPE_CHECK_ALL : "",
-ERROR_DECLARATION_DATA_TYPE_CHECK_NO_USE_C_TYPE : "There should be no use 
of int, unsigned, char, void, static, long in any .c, .h or .asl files",
+ERROR_DECLARATION_DATA_TYPE_CHECK_NO_USE_C_TYPE : "There should be no use 
of int, unsigned, char, void, long in any .c, .h or .asl files",
 ERROR_DECLARATION_DATA_TYPE_CHECK_IN_OUT_MODIFIER : """The modifiers IN, 
OUT, OPTIONAL, and UNALIGNED should be used only to qualify arguments to a 
function and should not appear in a data type declaration""",
 ERROR_DECLARATION_DATA_TYPE_CHECK_EFI_API_MODIFIER : "The EFIAPI modifier 
should be used at the entry of drivers, events, and member functions of 
protocols",
 ERROR_DECLARATION_DATA_TYPE_CHECK_ENUMERATED_TYPE : "Enumerated Type 
should have a 'typedef' and the name must be in capital letters",
diff --git a/BaseTools/Source/Python/Ecc/c.py b/BaseTools/Source/Python/Ecc/c.py
index 2a73da7cb8..a99b40a701 100644
--- a/BaseTools/Source/Python/Ecc/c.py
+++ b/BaseTools/Source/Python/Ecc/c.py
@@ -1859,7 +1859,7 @@ def CheckDeclNoUseCType(FullFileName):
where Model = %d
""" % (FileTable, DataClass.MODEL_IDENTIFIER_VARIABLE)
 ResultSet = Db.TblFile.Exec(SqlStatement)
-CTypeTuple = ('int', 'unsigned', 'char', 'void', 'static', 'long')
+CTypeTuple = ('int', 'unsigned', 'char', 'void', 'long')
 for Result in ResultSet:
 for Type in CTypeTuple:
 if PatternInModifier(Result[0], Type):
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#46776): https://edk2.groups.io/g/devel/message/46776
Mute This Topic: https://groups.io/mt/33136540/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH V3] [edk2-stable201908] BaseTools: Update incorrect variable name 'DataPile'

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2093

The PlatformAutoGen object has a DataPipe property but no DataPile property
So change the variable name 'DataPile' to 'DataPipe' in BuildReport.py

This patch is going to fix that issue.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/BuildReport.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/build/BuildReport.py 
b/BaseTools/Source/Python/build/BuildReport.py
index 9c12c01d2a..6b26f1c3b0 100644
--- a/BaseTools/Source/Python/build/BuildReport.py
+++ b/BaseTools/Source/Python/build/BuildReport.py
@@ -2142,7 +2142,7 @@ class PlatformReport(object):
 INFList = 
GlobalData.gFdfParser.Profile.InfDict[Pa.Arch]
 for InfName in INFList:
 InfClass = PathClass(NormPath(InfName), 
Wa.WorkspaceDir, Pa.Arch)
-Ma = ModuleAutoGen(Wa, InfClass, Pa.BuildTarget, 
Pa.ToolChain, Pa.Arch, Wa.MetaFile,Pa.DataPile)
+Ma = ModuleAutoGen(Wa, InfClass, Pa.BuildTarget, 
Pa.ToolChain, Pa.Arch, Wa.MetaFile, Pa.DataPipe)
 if Ma is None:
 continue
 if Ma not in ModuleAutoGenList:
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#46063): https://edk2.groups.io/g/devel/message/46063
Mute This Topic: https://groups.io/mt/32962507/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH V2] BaseTools: Update incorrect variable name 'DataPile'

[Fan, ZhijuX]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2093

The PlatformAutoGen object has a DataPipe property but no DataPile property
So change the variable name 'DataPile' to 'DataPipe' in BuildReport.py

This patch is going to fix that issue.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/BuildReport.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/build/BuildReport.py 
b/BaseTools/Source/Python/build/BuildReport.py
index 9c12c01d2a..6b26f1c3b0 100644
--- a/BaseTools/Source/Python/build/BuildReport.py
+++ b/BaseTools/Source/Python/build/BuildReport.py
@@ -2142,7 +2142,7 @@ class PlatformReport(object):
 INFList = 
GlobalData.gFdfParser.Profile.InfDict[Pa.Arch]
 for InfName in INFList:
 InfClass = PathClass(NormPath(InfName), 
Wa.WorkspaceDir, Pa.Arch)
-Ma = ModuleAutoGen(Wa, InfClass, Pa.BuildTarget, 
Pa.ToolChain, Pa.Arch, Wa.MetaFile,Pa.DataPile)
+Ma = ModuleAutoGen(Wa, InfClass, Pa.BuildTarget, 
Pa.ToolChain, Pa.Arch, Wa.MetaFile, Pa.DataPipe)
 if Ma is None:
 continue
 if Ma not in ModuleAutoGenList:
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#46034): https://edk2.groups.io/g/devel/message/46034
Mute This Topic: https://groups.io/mt/32942271/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools: Update incorrect variable name 'DataPile'

[Fan, ZhijuX]

The PlatformAutoGen object has a DataPipe property but no DataPile property
So change the variable name 'DataPile' to 'DataPipe' in BuildReport.py

This patch is going to fix that issue.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/BuildReport.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/build/BuildReport.py 
b/BaseTools/Source/Python/build/BuildReport.py
index 9c12c01d2a..6b26f1c3b0 100644
--- a/BaseTools/Source/Python/build/BuildReport.py
+++ b/BaseTools/Source/Python/build/BuildReport.py
@@ -2142,7 +2142,7 @@ class PlatformReport(object):
 INFList = 
GlobalData.gFdfParser.Profile.InfDict[Pa.Arch]
 for InfName in INFList:
 InfClass = PathClass(NormPath(InfName), 
Wa.WorkspaceDir, Pa.Arch)
-Ma = ModuleAutoGen(Wa, InfClass, Pa.BuildTarget, 
Pa.ToolChain, Pa.Arch, Wa.MetaFile,Pa.DataPile)
+Ma = ModuleAutoGen(Wa, InfClass, Pa.BuildTarget, 
Pa.ToolChain, Pa.Arch, Wa.MetaFile, Pa.DataPipe)
 if Ma is None:
 continue
 if Ma not in ModuleAutoGenList:
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#46033): https://edk2.groups.io/g/devel/message/46033
Mute This Topic: https://groups.io/mt/32942213/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH 2/2] BaseTools:Build cache generate the offset file for hii/vfr module

[Fan, ZhijuX]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1958

Current build cache cannot generate the offset file for hii/vfr module

Reproduce steps on OVMF:
1. Generate the build cache and save the build folder as Build1:
$ source edksetup.sh
$ build -p OvmfPkg/OvmfPkgIa32X64.dsc -a IA32 -a X64 -t GCC5 -b NOOPT
--hash --binary-destination=BinCache
$ mv Build/ Build1
2. Consume the build cache and save the build folder as Build2:
$ build -p OvmfPkg/OvmfPkgIa32X64.dsc -a IA32 -a X64 -t GCC5 -b NOOPT
--hash --binary-source=BinCache
$ mv Build Build2

Compare the hii/vfr module ffs folder, e.g. UiApp as below,
in the Build1 and Build2, you will find the *.offset, *.raw,
*.raw.txt, *.map files are missing.

This Patch is going to add these content to Cache

Cc: Bob Feng 
Cc: Liming Gao 
Cc: Steven Shi 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 ++--
 BaseTools/Source/Python/build/build.py   |  9 +-
 2 files changed, 43 insertions(+), 4 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py 
b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index 4bb72eb10d..d9baad9c62 100644
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -1244,6 +1244,19 @@ class ModuleAutoGen(AutoGen):
 fStringIO.close ()
 fInputfile.close ()
 return OutputName
+
+@cached_property
+def OutputFfsFile(self):
+retVal = set()
+FfsOutputDir = self.FfsOutputDir.replace('\\', '/').rstrip('/')
+for Root, Dirs, Files in os.walk(FfsOutputDir):
+for File in Files:
+if File.lower().endswith('.ffs') or 
File.lower().endswith('.offset') or File.lower().endswith('.raw') \
+or File.lower().endswith('.raw.txt') or 
File.lower().endswith('.map'):
+retVal.add(File)
+
+return retVal
+
 @cached_property
 def OutputFile(self):
 retVal = set()
@@ -1267,7 +1280,8 @@ class ModuleAutoGen(AutoGen):
 
 for Root, Dirs, Files in os.walk(FfsOutputDir):
 for File in Files:
-if File.lower().endswith('.ffs'):
+if File.lower().endswith('.ffs') or 
File.lower().endswith('.offset') or File.lower().endswith('.raw') \
+or File.lower().endswith('.raw.txt') or 
File.lower().endswith('.map'):
 retVal.add(File)
 
 return retVal
@@ -1607,6 +1621,23 @@ class ModuleAutoGen(AutoGen):
 CreateDirectory(destination_dir)
 CopyFileOnChange(File, destination_dir)
 
+def CopyFfsToCache(self):
+FfsDir = path.join(GlobalData.gBinCacheDest, 
self.PlatformInfo.OutputDir, self.BuildTarget + "_" + self.ToolChain, 
TAB_FV_DIRECTORY, "Ffs", self.Guid + self.Name)
+for File in self.OutputFfsFile:
+File = str(File)
+if not os.path.isabs(File):
+FfsFile = os.path.join(FfsDir, File)
+NewFile = os.path.join(self.FfsOutputDir, File)
+File = NewFile
+else:
+sub_dir = os.path.relpath(File, self.FfsOutputDir)
+destination_file = os.path.join(FfsDir, sub_dir)
+FfsFile = os.path.dirname(destination_file)
+if os.path.exists(File) and not os.path.exists(FfsFile):
+if File.lower().endswith('.ffs') or 
File.lower().endswith('.offset') or File.lower().endswith('.raw') \
+or File.lower().endswith('.raw.txt') or 
File.lower().endswith('.map'):
+self.CacheCopyFile(FfsDir, self.FfsOutputDir, File)
+
 def CopyModuleToCache(self):
 FileDir = path.join(GlobalData.gBinCacheDest, self.PlatformInfo.Name, 
self.BuildTarget + "_" + self.ToolChain, self.Arch, self.SourceDir, 
self.MetaFile.BaseName)
 FfsDir = path.join(GlobalData.gBinCacheDest, 
self.PlatformInfo.OutputDir, self.BuildTarget + "_" + self.ToolChain, 
TAB_FV_DIRECTORY, "Ffs", self.Guid + self.Name)
@@ -1628,7 +1659,8 @@ class ModuleAutoGen(AutoGen):
 NewFile = os.path.join(self.FfsOutputDir, File)
 File = NewFile
 if os.path.exists(File):
-if File.endswith('.ffs'):
+if File.lower().endswith('.ffs') or 
File.lower().endswith('.offset') or File.lower().endswith('.raw') \
+or File.lower().endswith('.raw.txt') or 
File.lower().endswith('.map'):
 self.CacheCopyFile(FfsDir, self.FfsOutputDir, File)
 else:
 self.CacheCopyFile(FileDir, self.OutputDir, File)
@@ -1658,7 +1690,7 @@ class ModuleAutoGen(AutoGen):
 else:
 File = path.join(root, f)
 self.CacheCopyFile(self.OutputDir, FileDir, 
File)
-if GlobalData.gEnableGenfdsMultiThread and 

[edk2-devel] [PATCH 1/2] BaseTools:Fix GenFds multi-thread build fails if enable build cache

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1923

The GenFds multi-thread will build fail if enable the build cache.
1. First, produce the build cache:
edk2>build -p OvmfPkg\OvmfPkgIa32X64.dsc -a IA32 -a X64 -t VS2015x86 -n 5
--genfds-multi-thread --hash --binary-destination=BinCache

2. Remove the build folder:
edk2>rmdir Build /s /q

3. Clean build with build cache and GenFds multi-thread enabled together:
edk2>build -p OvmfPkg\OvmfPkgIa32X64.dsc -a IA32 -a X64 -t VS2015x86 -n 5
--genfds-multi-thread --hash --binary-source=BinCache

If disable GenFds multi-thread, the build cache can work well

This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Cc: Steven Shi 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 41 +---
 1 file changed, 30 insertions(+), 11 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py 
b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index 36bbaffa56..4bb72eb10d 100644
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -1249,6 +1249,7 @@ class ModuleAutoGen(AutoGen):
 retVal = set()
 OutputDir = self.OutputDir.replace('\\', '/').strip('/')
 DebugDir = self.DebugDir.replace('\\', '/').strip('/')
+FfsOutputDir = self.FfsOutputDir.replace('\\', '/').rstrip('/')
 for Item in self.CodaTargetList:
 File = Item.Target.Path.replace('\\', 
'/').strip('/').replace(DebugDir, '').replace(OutputDir, '').strip('/')
 retVal.add(File)
@@ -1264,6 +1265,11 @@ class ModuleAutoGen(AutoGen):
 if File.lower().endswith('.pdb'):
 retVal.add(File)
 
+for Root, Dirs, Files in os.walk(FfsOutputDir):
+for File in Files:
+if File.lower().endswith('.ffs'):
+retVal.add(File)
+
 return retVal
 
 ## Create AsBuilt INF file the module
@@ -1594,8 +1600,16 @@ class ModuleAutoGen(AutoGen):
 
 self.IsAsBuiltInfCreated = True
 
+def CacheCopyFile(self, OriginDir, CopyDir, File):
+sub_dir = os.path.relpath(File, CopyDir)
+destination_file = os.path.join(OriginDir, sub_dir)
+destination_dir = os.path.dirname(destination_file)
+CreateDirectory(destination_dir)
+CopyFileOnChange(File, destination_dir)
+
 def CopyModuleToCache(self):
 FileDir = path.join(GlobalData.gBinCacheDest, self.PlatformInfo.Name, 
self.BuildTarget + "_" + self.ToolChain, self.Arch, self.SourceDir, 
self.MetaFile.BaseName)
+FfsDir = path.join(GlobalData.gBinCacheDest, 
self.PlatformInfo.OutputDir, self.BuildTarget + "_" + self.ToolChain, 
TAB_FV_DIRECTORY, "Ffs", self.Guid + self.Name)
 CreateDirectory (FileDir)
 HashFile = path.join(self.BuildDir, self.Name + '.hash')
 if os.path.exists(HashFile):
@@ -1609,13 +1623,15 @@ class ModuleAutoGen(AutoGen):
 for File in self.OutputFile:
 File = str(File)
 if not os.path.isabs(File):
-File = os.path.join(self.OutputDir, File)
+NewFile = os.path.join(self.OutputDir, File)
+if not os.path.exists(NewFile):
+NewFile = os.path.join(self.FfsOutputDir, File)
+File = NewFile
 if os.path.exists(File):
-sub_dir = os.path.relpath(File, self.OutputDir)
-destination_file = os.path.join(FileDir, sub_dir)
-destination_dir = os.path.dirname(destination_file)
-CreateDirectory(destination_dir)
-CopyFileOnChange(File, destination_dir)
+if File.endswith('.ffs'):
+self.CacheCopyFile(FfsDir, self.FfsOutputDir, File)
+else:
+self.CacheCopyFile(FileDir, self.OutputDir, File)
 
 def AttemptModuleCacheCopy(self):
 # If library or Module is binary do not skip by hash
@@ -1626,6 +1642,7 @@ class ModuleAutoGen(AutoGen):
 if '.inc' in str(f_ext):
 return False
 FileDir = path.join(GlobalData.gBinCacheSource, 
self.PlatformInfo.Name, self.BuildTarget + "_" + self.ToolChain, self.Arch, 
self.SourceDir, self.MetaFile.BaseName)
+FfsDir = path.join(GlobalData.gBinCacheSource, 
self.PlatformInfo.OutputDir, self.BuildTarget + "_" + self.ToolChain, 
TAB_FV_DIRECTORY, "Ffs", self.Guid + self.Name)
 HashFile = path.join(FileDir, self.Name + '.hash')
 if os.path.exists(HashFile):
 f = open(HashFile, 'r')
@@ -1640,11 +1657,13 @@ class ModuleAutoGen(AutoGen):
 CopyFileOnChange(HashFile, self.BuildDir)
 else:
 File = path.join(root, f)
-sub_dir = os.path.relpath(File, FileDir)
-destination_file = 

[edk2-devel] [PATCH] Build Spec:Add Module Arch to BuildRebuild Module Section

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2022

Added arch output to build report so it's easy to tell
which arch is being generated. Code change
https://bugzilla.tianocore.org/show_bug.cgi?id=2016
will be done.The spec change is also required.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 13_build_reports/138_module_section.md | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/13_build_reports/138_module_section.md 
b/13_build_reports/138_module_section.md
index d457c24..6f7edc5 100644
--- a/13_build_reports/138_module_section.md
+++ b/13_build_reports/138_module_section.md
@@ -38,11 +38,12 @@ sections are listed according to their DSC position.
 
 ### 13.8.1 Module Section Summary
 
-This sub-section lists the module basic information: Module name, INF file
-path, File GUID, Size, hash value, module build time stamp, module build time
-and driver type.
+This sub-section lists the module basic information: Module name, Module Arch,
+INF file path, File GUID, Size, hash value, module build time stamp, module
+build time and driver type.
 
 * Module Name : %`BASE_NAME` in INF `[Defines]` section%
+* Module Arch : %Architecture of current module%
 * Module INF Path : %Path of Module INF file%
 * File GUID : %`FILE_GUID` in INF `[Defines]` section%
 * Size : %Module EFI image size%
@@ -70,6 +71,7 @@ Volume 3 of the PI Specification (Table 3 Defined File Types).
 >==<
 Module Summary
 Module Name:SmbiosDxe
+Module Arch:X64
 Module INF Path:MdeModule\Universal\SmbiosDxe\SmbiosDxe.inf
 File GUID:  F9D88642-0737-49BC-81B5-6889CD57D9EA
 Size:   0x7000 (28.00K)
@@ -88,6 +90,7 @@ Driver Type:0x7 (DRIVER)
 >==<
 Module Summary
 Module Name:EbcDxe
+Module Arch:X64
 Module INF Path:MdeModule\Universal\EbcDxe\EbcDxe.inf
 File GUID:  13AC6DD0-73D0-11D4-B06B-00AA00BD6DE7
 Size:   0x9000 (36.00K)
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44429): https://edk2.groups.io/g/devel/message/44429
Mute This Topic: https://groups.io/mt/32605502/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH V2] BaseTools:Added arch output to build report

[Fan, ZhijuX]

From: Matthew Carlson 

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2016

Added arch output to build report so it's easy to tell
which arch is being generated. Useful when multiple versions
of a single module is being emitted for multiple archs.

This patch is going to Added arch output

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/BuildReport.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/BaseTools/Source/Python/build/BuildReport.py 
b/BaseTools/Source/Python/build/BuildReport.py
index a3eb3b2383..b4189240e1 100644
--- a/BaseTools/Source/Python/build/BuildReport.py
+++ b/BaseTools/Source/Python/build/BuildReport.py
@@ -558,6 +558,7 @@ class ModuleReport(object):
 def __init__(self, M, ReportType):
 self.ModuleName = M.Module.BaseName
 self.ModuleInfPath = M.MetaFile.File
+self.ModuleArch = M.Arch
 self.FileGuid = M.Guid
 self.Size = 0
 self.BuildTimeStamp = None
@@ -668,6 +669,7 @@ class ModuleReport(object):
 
 FileWrite(File, "Module Summary")
 FileWrite(File, "Module Name:  %s" % self.ModuleName)
+FileWrite(File, "Module Arch:  %s" % self.ModuleArch)
 FileWrite(File, "Module INF Path:  %s" % self.ModuleInfPath)
 FileWrite(File, "File GUID:%s" % self.FileGuid)
 if self.Size:
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44361): https://edk2.groups.io/g/devel/message/44361
Mute This Topic: https://groups.io/mt/32594597/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH V2] BaseTools:Updata the output encoding of the Popen function

[Fan, ZhijuX]

From: Bob Feng 

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2015

Not all output works in utf-8, so change the encoding to
the default

This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Common/Misc.py  | 2 +-
 BaseTools/Source/Python/Common/VpdInfoFile.py   | 2 +-
 BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py  | 2 +-
 .../Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py| 6 +++---
 BaseTools/Source/Python/Workspace/DscBuildData.py   | 2 +-
 5 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/Common/Misc.py 
b/BaseTools/Source/Python/Common/Misc.py
index 9a63463913..36bd2fe364 100644
--- a/BaseTools/Source/Python/Common/Misc.py
+++ b/BaseTools/Source/Python/Common/Misc.py
@@ -1092,7 +1092,7 @@ def ParseFieldValue (Value):
 p.stderr.close()
 if err:
 raise BadExpression("DevicePath: %s" % str(err))
-out = out.decode(encoding='utf-8', errors='ignore')
+out = out.decode()
 Size = len(out.split())
 out = ','.join(out.split())
 return '{' + out + '}', Size
diff --git a/BaseTools/Source/Python/Common/VpdInfoFile.py 
b/BaseTools/Source/Python/Common/VpdInfoFile.py
index bc69a9010d..4249b9f899 100644
--- a/BaseTools/Source/Python/Common/VpdInfoFile.py
+++ b/BaseTools/Source/Python/Common/VpdInfoFile.py
@@ -243,7 +243,7 @@ def CallExtenalBPDGTool(ToolPath, VpdFileName):
 except Exception as X:
 EdkLogger.error("BPDG", BuildToolError.COMMAND_FAILURE, 
ExtraData=str(X))
 (out, error) = PopenObject.communicate()
-print(out.decode(encoding='utf-8', errors='ignore'))
+print(out.decode())
 while PopenObject.returncode is None :
 PopenObject.wait()
 
diff --git a/BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py 
b/BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py
index 5630df55df..5d4c3a8599 100644
--- a/BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py
+++ b/BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py
@@ -116,7 +116,7 @@ if __name__ == '__main__':
   if Process.returncode != 0:
 print('ERROR: Open SSL command not available.  Please verify PATH or set 
OPENSSL_PATH')
 sys.exit(Process.returncode)
-  print(Version[0].decode(encoding='utf-8', errors='ignore'))
+  print(Version[0].decode())
 
   #
   # Read input file into a buffer and save input filename
diff --git 
a/BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py 
b/BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py
index f9aed2bf86..6c9b8c464e 100644
--- a/BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py
+++ b/BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py
@@ -78,7 +78,7 @@ if __name__ == '__main__':
   if Process.returncode != 0:
 print('ERROR: Open SSL command not available.  Please verify PATH or set 
OPENSSL_PATH')
 sys.exit(Process.returncode)
-  print(Version[0].decode(encoding='utf-8', errors='ignore'))
+  print(Version[0].decode())
 
   args.PemFileName = []
 
@@ -119,7 +119,7 @@ if __name__ == '__main__':
 # Extract public key from private key into STDOUT
 #
 Process = subprocess.Popen('%s rsa -in %s -modulus -noout' % 
(OpenSslCommand, Item), stdout=subprocess.PIPE, stderr=subprocess.PIPE, 
shell=True)
-PublicKeyHexString = Process.communicate()[0].decode(encoding='utf-8', 
errors='ignore').split(b'=')[1].strip()
+PublicKeyHexString = 
Process.communicate()[0].decode().split(b'=')[1].strip()
 if Process.returncode != 0:
   print('ERROR: Unable to extract public key from private key')
   sys.exit(Process.returncode)
@@ -132,7 +132,7 @@ if __name__ == '__main__':
 #
 Process = subprocess.Popen('%s dgst -sha256 -binary' % (OpenSslCommand), 
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, 
shell=True)
 Process.stdin.write (PublicKey)
-PublicKeyHash = PublicKeyHash + 
Process.communicate()[0].decode(encoding='utf-8', errors='ignore')
+PublicKeyHash = PublicKeyHash + Process.communicate()[0].decode()
 if Process.returncode != 0:
   print('ERROR: Unable to extract SHA 256 hash of public key')
   sys.exit(Process.returncode)
diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py 
b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 985f877525..620e48fa7f 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -1735,7 +1735,7 @@ class DscBuildData(PlatformBuildClassObject):
 except:
 EdkLogger.error('Build', COMMAND_FAILURE, 'Can not execute 
command: %s' % Command)
 Result = Process.communicate()
-return Process.returncode, Result[0].decode(encoding='utf-8', 
errors='ignore'), Result[1].decode(encoding='utf-8', errors='ignore')
+return 

[edk2-devel] [PATCH V2] BaseTools:Add extra debugging message

[Fan, ZhijuX]

From: Max Knutsen 

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2014

Add extra debugging to improve error identification.
Error while processing file if the file is read incorrectly

This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/StrGather.py | 16 ++--
 BaseTools/Source/Python/Trim/Trim.py |  4 +++-
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/StrGather.py 
b/BaseTools/Source/Python/AutoGen/StrGather.py
index 2e4671a433..eed30388be 100644
--- a/BaseTools/Source/Python/AutoGen/StrGather.py
+++ b/BaseTools/Source/Python/AutoGen/StrGather.py
@@ -526,12 +526,16 @@ def SearchString(UniObjectClass, FileList, 
IsCompatibleMode):
 return UniObjectClass
 
 for File in FileList:
-if os.path.isfile(File):
-Lines = open(File, 'r')
-for Line in Lines:
-for StrName in STRING_TOKEN.findall(Line):
-EdkLogger.debug(EdkLogger.DEBUG_5, "Found string 
identifier: " + StrName)
-UniObjectClass.SetStringReferenced(StrName)
+try:
+if os.path.isfile(File):
+Lines = open(File, 'r')
+for Line in Lines:
+for StrName in STRING_TOKEN.findall(Line):
+EdkLogger.debug(EdkLogger.DEBUG_5, "Found string 
identifier: " + StrName)
+UniObjectClass.SetStringReferenced(StrName)
+except:
+EdkLogger.error("UnicodeStringGather", AUTOGEN_ERROR, 
"SearchString: Error while processing file", File=File, RaiseError=False)
+raise
 
 UniObjectClass.ReToken()
 
diff --git a/BaseTools/Source/Python/Trim/Trim.py 
b/BaseTools/Source/Python/Trim/Trim.py
index 43119bd7ff..8767b67f7e 100644
--- a/BaseTools/Source/Python/Trim/Trim.py
+++ b/BaseTools/Source/Python/Trim/Trim.py
@@ -73,8 +73,10 @@ def TrimPreprocessedFile(Source, Target, ConvertHex, 
TrimLong):
 try:
 with open(Source, "r") as File:
 Lines = File.readlines()
-except:
+except IOError:
 EdkLogger.error("Trim", FILE_OPEN_FAILURE, ExtraData=Source)
+expect:
+EdkLogger.error("Trim", AUTOGEN_ERROR, "TrimPreprocessedFile: Error 
while processing file", File=Source)
 
 PreprocessedFile = ""
 InjectedFile = ""
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44359): https://edk2.groups.io/g/devel/message/44359
Mute This Topic: https://groups.io/mt/32594584/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH V2] BaseTools: replace the chinese quotation mark with unicode "

[Fan, ZhijuX]

From: Sean Brogan 

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2013

chinese quotation mark are used in the file
This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Pkcs7Sign/Readme.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/Pkcs7Sign/Readme.md 
b/BaseTools/Source/Python/Pkcs7Sign/Readme.md
index 5315b7fca4..a7b9caf52c 100644
--- a/BaseTools/Source/Python/Pkcs7Sign/Readme.md
+++ b/BaseTools/Source/Python/Pkcs7Sign/Readme.md
@@ -10,7 +10,7 @@ NOTE: Below steps are required for Windows. Linux may already 
have the OPENSSL e
 set OPENSSL_HOME=c:\home\openssl\openssl-[version]
 set OPENSSL_CONF=%OPENSSL_HOME%\apps\openssl.cnf
 
-When a user uses OpenSSL (req or ca command) to generate the certificates, 
OpenSSL will use the openssl.cnf file as the configuration data (can use 
“-config path/to/openssl.cnf” to describe the specific config file).
+When a user uses OpenSSL (req or ca command) to generate the certificates, 
OpenSSL will use the openssl.cnf file as the configuration data (can use 
"-config path/to/openssl.cnf" to describe the specific config file).
 
 The user need check the openssl.cnf file, to find your CA path setting, e.g. 
check if the path exists in [ CA_default ] section.
 
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44358): https://edk2.groups.io/g/devel/message/44358
Mute This Topic: https://groups.io/mt/32594579/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH] BaseTools:Updata the output encoding of the Popen function

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2015

Not all output works in utf-8, so change the encoding to
the default

This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Common/Misc.py  | 2 +-
 BaseTools/Source/Python/Common/VpdInfoFile.py   | 2 +-
 BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py  | 2 +-
 .../Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py| 6 +++---
 BaseTools/Source/Python/Workspace/DscBuildData.py   | 2 +-
 5 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/Common/Misc.py 
b/BaseTools/Source/Python/Common/Misc.py
index 9a63463913..36bd2fe364 100644
--- a/BaseTools/Source/Python/Common/Misc.py
+++ b/BaseTools/Source/Python/Common/Misc.py
@@ -1092,7 +1092,7 @@ def ParseFieldValue (Value):
 p.stderr.close()
 if err:
 raise BadExpression("DevicePath: %s" % str(err))
-out = out.decode(encoding='utf-8', errors='ignore')
+out = out.decode()
 Size = len(out.split())
 out = ','.join(out.split())
 return '{' + out + '}', Size
diff --git a/BaseTools/Source/Python/Common/VpdInfoFile.py 
b/BaseTools/Source/Python/Common/VpdInfoFile.py
index bc69a9010d..4249b9f899 100644
--- a/BaseTools/Source/Python/Common/VpdInfoFile.py
+++ b/BaseTools/Source/Python/Common/VpdInfoFile.py
@@ -243,7 +243,7 @@ def CallExtenalBPDGTool(ToolPath, VpdFileName):
 except Exception as X:
 EdkLogger.error("BPDG", BuildToolError.COMMAND_FAILURE, 
ExtraData=str(X))
 (out, error) = PopenObject.communicate()
-print(out.decode(encoding='utf-8', errors='ignore'))
+print(out.decode())
 while PopenObject.returncode is None :
 PopenObject.wait()
 
diff --git a/BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py 
b/BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py
index 5630df55df..5d4c3a8599 100644
--- a/BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py
+++ b/BaseTools/Source/Python/Pkcs7Sign/Pkcs7Sign.py
@@ -116,7 +116,7 @@ if __name__ == '__main__':
   if Process.returncode != 0:
 print('ERROR: Open SSL command not available.  Please verify PATH or set 
OPENSSL_PATH')
 sys.exit(Process.returncode)
-  print(Version[0].decode(encoding='utf-8', errors='ignore'))
+  print(Version[0].decode())
 
   #
   # Read input file into a buffer and save input filename
diff --git 
a/BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py 
b/BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py
index f9aed2bf86..6c9b8c464e 100644
--- a/BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py
+++ b/BaseTools/Source/Python/Rsa2048Sha256Sign/Rsa2048Sha256GenerateKeys.py
@@ -78,7 +78,7 @@ if __name__ == '__main__':
   if Process.returncode != 0:
 print('ERROR: Open SSL command not available.  Please verify PATH or set 
OPENSSL_PATH')
 sys.exit(Process.returncode)
-  print(Version[0].decode(encoding='utf-8', errors='ignore'))
+  print(Version[0].decode())
 
   args.PemFileName = []
 
@@ -119,7 +119,7 @@ if __name__ == '__main__':
 # Extract public key from private key into STDOUT
 #
 Process = subprocess.Popen('%s rsa -in %s -modulus -noout' % 
(OpenSslCommand, Item), stdout=subprocess.PIPE, stderr=subprocess.PIPE, 
shell=True)
-PublicKeyHexString = Process.communicate()[0].decode(encoding='utf-8', 
errors='ignore').split(b'=')[1].strip()
+PublicKeyHexString = 
Process.communicate()[0].decode().split(b'=')[1].strip()
 if Process.returncode != 0:
   print('ERROR: Unable to extract public key from private key')
   sys.exit(Process.returncode)
@@ -132,7 +132,7 @@ if __name__ == '__main__':
 #
 Process = subprocess.Popen('%s dgst -sha256 -binary' % (OpenSslCommand), 
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, 
shell=True)
 Process.stdin.write (PublicKey)
-PublicKeyHash = PublicKeyHash + 
Process.communicate()[0].decode(encoding='utf-8', errors='ignore')
+PublicKeyHash = PublicKeyHash + Process.communicate()[0].decode()
 if Process.returncode != 0:
   print('ERROR: Unable to extract SHA 256 hash of public key')
   sys.exit(Process.returncode)
diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py 
b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 985f877525..620e48fa7f 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -1735,7 +1735,7 @@ class DscBuildData(PlatformBuildClassObject):
 except:
 EdkLogger.error('Build', COMMAND_FAILURE, 'Can not execute 
command: %s' % Command)
 Result = Process.communicate()
-return Process.returncode, Result[0].decode(encoding='utf-8', 
errors='ignore'), Result[1].decode(encoding='utf-8', errors='ignore')
+return Process.returncode, 

[edk2-devel] [PATCH] BaseTools:Added arch output to build report

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2016

Added arch output to build report so it's easy to tell
which arch is being generated. Useful when multiple versions
of a single module is being emitted for multiple archs.

This patch is going to Added arch output

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/build/BuildReport.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/BaseTools/Source/Python/build/BuildReport.py 
b/BaseTools/Source/Python/build/BuildReport.py
index a3eb3b2383..b4189240e1 100644
--- a/BaseTools/Source/Python/build/BuildReport.py
+++ b/BaseTools/Source/Python/build/BuildReport.py
@@ -558,6 +558,7 @@ class ModuleReport(object):
 def __init__(self, M, ReportType):
 self.ModuleName = M.Module.BaseName
 self.ModuleInfPath = M.MetaFile.File
+self.ModuleArch = M.Arch
 self.FileGuid = M.Guid
 self.Size = 0
 self.BuildTimeStamp = None
@@ -668,6 +669,7 @@ class ModuleReport(object):
 
 FileWrite(File, "Module Summary")
 FileWrite(File, "Module Name:  %s" % self.ModuleName)
+FileWrite(File, "Module Arch:  %s" % self.ModuleArch)
 FileWrite(File, "Module INF Path:  %s" % self.ModuleInfPath)
 FileWrite(File, "File GUID:%s" % self.FileGuid)
 if self.Size:
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44346): https://edk2.groups.io/g/devel/message/44346
Mute This Topic: https://groups.io/mt/32593782/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:Add extra debugging message

[Fan, ZhijuX]

Add extra debugging to improve error identification.
Error while processing file if the file is read incorrectly

This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/StrGather.py | 16 ++--
 BaseTools/Source/Python/Trim/Trim.py |  4 +++-
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/StrGather.py 
b/BaseTools/Source/Python/AutoGen/StrGather.py
index 2e4671a433..eed30388be 100644
--- a/BaseTools/Source/Python/AutoGen/StrGather.py
+++ b/BaseTools/Source/Python/AutoGen/StrGather.py
@@ -526,12 +526,16 @@ def SearchString(UniObjectClass, FileList, 
IsCompatibleMode):
 return UniObjectClass
 
 for File in FileList:
-if os.path.isfile(File):
-Lines = open(File, 'r')
-for Line in Lines:
-for StrName in STRING_TOKEN.findall(Line):
-EdkLogger.debug(EdkLogger.DEBUG_5, "Found string 
identifier: " + StrName)
-UniObjectClass.SetStringReferenced(StrName)
+try:
+if os.path.isfile(File):
+Lines = open(File, 'r')
+for Line in Lines:
+for StrName in STRING_TOKEN.findall(Line):
+EdkLogger.debug(EdkLogger.DEBUG_5, "Found string 
identifier: " + StrName)
+UniObjectClass.SetStringReferenced(StrName)
+except:
+EdkLogger.error("UnicodeStringGather", AUTOGEN_ERROR, 
"SearchString: Error while processing file", File=File, RaiseError=False)
+raise
 
 UniObjectClass.ReToken()
 
diff --git a/BaseTools/Source/Python/Trim/Trim.py 
b/BaseTools/Source/Python/Trim/Trim.py
index 43119bd7ff..8767b67f7e 100644
--- a/BaseTools/Source/Python/Trim/Trim.py
+++ b/BaseTools/Source/Python/Trim/Trim.py
@@ -73,8 +73,10 @@ def TrimPreprocessedFile(Source, Target, ConvertHex, 
TrimLong):
 try:
 with open(Source, "r") as File:
 Lines = File.readlines()
-except:
+except IOError:
 EdkLogger.error("Trim", FILE_OPEN_FAILURE, ExtraData=Source)
+expect:
+EdkLogger.error("Trim", AUTOGEN_ERROR, "TrimPreprocessedFile: Error 
while processing file", File=Source)
 
 PreprocessedFile = ""
 InjectedFile = ""
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44344): https://edk2.groups.io/g/devel/message/44344
Mute This Topic: https://groups.io/mt/32593771/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools: replace the chinese quotation mark with unicode "

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2013

chinese quotation mark are used in the file
This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/Pkcs7Sign/Readme.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/Pkcs7Sign/Readme.md 
b/BaseTools/Source/Python/Pkcs7Sign/Readme.md
index 5315b7fca4..a7b9caf52c 100644
--- a/BaseTools/Source/Python/Pkcs7Sign/Readme.md
+++ b/BaseTools/Source/Python/Pkcs7Sign/Readme.md
@@ -10,7 +10,7 @@ NOTE: Below steps are required for Windows. Linux may already 
have the OPENSSL e
 set OPENSSL_HOME=c:\home\openssl\openssl-[version]
 set OPENSSL_CONF=%OPENSSL_HOME%\apps\openssl.cnf
 
-When a user uses OpenSSL (req or ca command) to generate the certificates, 
OpenSSL will use the openssl.cnf file as the configuration data (can use 
“-config path/to/openssl.cnf” to describe the specific config file).
+When a user uses OpenSSL (req or ca command) to generate the certificates, 
OpenSSL will use the openssl.cnf file as the configuration data (can use 
"-config path/to/openssl.cnf" to describe the specific config file).
 
 The user need check the openssl.cnf file, to find your CA path setting, e.g. 
check if the path exists in [ CA_default ] section.
 
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44343): https://edk2.groups.io/g/devel/message/44343
Mute This Topic: https://groups.io/mt/32593754/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:Fix GenFds multi-thread build fails if enable build cache

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1923

The GenFds multi-thread will build fail if enable the build cache.
1. First, produce the build cache:
edk2>build -p OvmfPkg\OvmfPkgIa32X64.dsc -a IA32 -a X64 -t VS2015x86 -n 5
--genfds-multi-thread --hash --binary-destination=BinCache

2. Remove the build folder:
edk2>rmdir Build /s /q

3. Clean build with build cache and GenFds multi-thread enabled together:
edk2>build -p OvmfPkg\OvmfPkgIa32X64.dsc -a IA32 -a X64 -t VS2015x86 -n 5
--genfds-multi-thread --hash --binary-source=BinCache

If disable GenFds multi-thread, the build cache can work well

This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Cc: Shi, Steven 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 41 ++
 1 file changed, 30 insertions(+), 11 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py 
b/BaseTools/Source/Python/AutoGen/AutoGen.py
index e8e09dc8a3..77eca79fc2 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -3584,6 +3584,7 @@ class ModuleAutoGen(AutoGen):
 retVal = set()
 OutputDir = self.OutputDir.replace('\\', '/').strip('/')
 DebugDir = self.DebugDir.replace('\\', '/').strip('/')
+FfsOutputDir = self.FfsOutputDir.replace('\\', '/').rstrip('/')
 for Item in self.CodaTargetList:
 File = Item.Target.Path.replace('\\', 
'/').strip('/').replace(DebugDir, '').replace(OutputDir, '').strip('/')
 retVal.add(File)
@@ -3599,6 +3600,11 @@ class ModuleAutoGen(AutoGen):
 if File.lower().endswith('.pdb'):
 retVal.add(File)
 
+for Root, Dirs, Files in os.walk(FfsOutputDir):
+for File in Files:
+if File.lower().endswith('.ffs'):
+retVal.add(File)
+
 return retVal
 
 ## Create AsBuilt INF file the module
@@ -3929,8 +3935,16 @@ class ModuleAutoGen(AutoGen):
 
 self.IsAsBuiltInfCreated = True
 
+def CacheCopyFile(self, OriginDir, CopyDir, File):
+sub_dir = os.path.relpath(File, CopyDir)
+destination_file = os.path.join(OriginDir, sub_dir)
+destination_dir = os.path.dirname(destination_file)
+CreateDirectory(destination_dir)
+CopyFileOnChange(File, destination_dir)
+
 def CopyModuleToCache(self):
 FileDir = path.join(GlobalData.gBinCacheDest, 
self.PlatformInfo.OutputDir, self.BuildTarget + "_" + self.ToolChain, 
self.Arch, self.SourceDir, self.MetaFile.BaseName)
+FfsDir = path.join(GlobalData.gBinCacheDest, 
self.PlatformInfo.OutputDir, self.BuildTarget + "_" + self.ToolChain, 
TAB_FV_DIRECTORY, "Ffs", self.Guid + self.Name)
 CreateDirectory (FileDir)
 HashFile = path.join(self.BuildDir, self.Name + '.hash')
 if os.path.exists(HashFile):
@@ -3946,13 +3960,15 @@ class ModuleAutoGen(AutoGen):
 for File in self.OutputFile:
 File = str(File)
 if not os.path.isabs(File):
-File = os.path.join(self.OutputDir, File)
+NewFile = os.path.join(self.OutputDir, File)
+if not os.path.exists(NewFile):
+NewFile = os.path.join(self.FfsOutputDir, File)
+File = NewFile
 if os.path.exists(File):
-sub_dir = os.path.relpath(File, self.OutputDir)
-destination_file = os.path.join(FileDir, sub_dir)
-destination_dir = os.path.dirname(destination_file)
-CreateDirectory(destination_dir)
-CopyFileOnChange(File, destination_dir)
+if File.endswith('.ffs'):
+self.CacheCopyFile(FfsDir, self.FfsOutputDir, File)
+else:
+self.CacheCopyFile(FileDir, self.OutputDir, File)
 
 def AttemptModuleCacheCopy(self):
 # If library or Module is binary do not skip by hash
@@ -3963,6 +3979,7 @@ class ModuleAutoGen(AutoGen):
 if '.inc' in str(f_ext):
 return False
 FileDir = path.join(GlobalData.gBinCacheSource, 
self.PlatformInfo.OutputDir, self.BuildTarget + "_" + self.ToolChain, 
self.Arch, self.SourceDir, self.MetaFile.BaseName)
+FfsDir = path.join(GlobalData.gBinCacheSource, 
self.PlatformInfo.OutputDir, self.BuildTarget + "_" + self.ToolChain, 
TAB_FV_DIRECTORY, "Ffs", self.Guid + self.Name)
 HashFile = path.join(FileDir, self.Name + '.hash')
 if os.path.exists(HashFile):
 f = open(HashFile, 'r')
@@ -3977,11 +3994,13 @@ class ModuleAutoGen(AutoGen):
 CopyFileOnChange(HashFile, self.BuildDir)
 else:
 File = path.join(root, f)
-sub_dir = os.path.relpath(File, FileDir)
-destination_file = 
os.path.join(self.OutputDir, sub_dir)

Re: [edk2-devel] [PATCH] BaseTools:Linux changes the way the latest version is judged

[Fan, ZhijuX]

Hi:

You're right, but at least we won't have to worry about this until “python3.10”

· And we're comparing directory names, "Python 3.6.6" doesn't appear in 
the Python version we found

· We find python applications using “whereis python”   
“/usr/bin/python3.6” “/usr/local/bin/python3.6”

· Later we can compare the form of string length to add a layer of 
judgment, but we don't need it now.


Any question, please let me know. Thanks.

Best Regards
Fan Zhiju



From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of 
rebe...@bluestop.org
Sent: Wednesday, June 26, 2019 11:48 PM
To: devel@edk2.groups.io; Fan, ZhijuX 
Cc: Gao, Liming ; Feng, Bob C 
Subject: Re: FW: [edk2-devel] [PATCH] BaseTools:Linux changes the way the 
latest version is judged

On 2019-06-25 22:46, Fan, ZhijuX wrote:

Hi:
Python3.6 and python3.7
After the code removes some common values “python”, we compare “3.6” and “3.7”
origin_version = “3.6”, python_version=”3.7”

I'm going to change it to
if [[ "$origin_version" < "$python_version" ]]; then





That's fine for now, hopefully it'll keep working as long as it needs to.

But once you're comparing for example 3.7 and 3.12:

[bcran@photon ~]$ if [[ "3.7" > "3.12" ]]; then echo "wrong!"; else echo 
"correct!"; fi
wrong!

--
Rebecca Cran


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#42990): https://edk2.groups.io/g/devel/message/42990
Mute This Topic: https://groups.io/mt/32154778/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH V2] BaseTools:Add the Judgment Method of "--exclude"

[Fan, ZhijuX]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1944

--exclude cannot be used under certain circumstances
1.The value of the parameter USES an absolute path
2.The value of Exclude parameters is based on the value
  of the Path parameter
Neither of these approaches currently works

This patch is going to fix that issue.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
keep original behavior on 
--exclude for the directory name or file name.

 BaseTools/Scripts/FormatDosFiles.py | 18 +-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Scripts/FormatDosFiles.py 
b/BaseTools/Scripts/FormatDosFiles.py
index d9a880f299..3150bef3c2 100644
--- a/BaseTools/Scripts/FormatDosFiles.py
+++ b/BaseTools/Scripts/FormatDosFiles.py
@@ -20,7 +20,7 @@ import copy
 
 __prog__= 'FormatDosFiles'
 __version__ = '%s Version %s' % (__prog__, '0.10 ')
-__copyright__   = 'Copyright (c) 2018, Intel Corporation. All rights reserved.'
+__copyright__   = 'Copyright (c) 2018-2019, Intel Corporation. All rights 
reserved.'
 __description__ = 'Convert source files to meet the EDKII C Coding Standards 
Specification.\n'
 DEFAULT_EXT_LIST = ['.h', '.c', '.nasm', '.nasmb', '.asm', '.S', '.inf', 
'.dec', '.dsc', '.fdf', '.uni', '.asl', '.aslc', '.vfr', '.idf', '.txt', 
'.bat', '.py']
 
@@ -46,10 +46,26 @@ def FormatFile(FilePath, Args):
 def FormatFilesInDir(DirPath, ExtList, Args):
 
 FileList = []
+ExcludeDir = DirPath
 for DirPath, DirNames, FileNames in os.walk(DirPath):
 if Args.Exclude:
 DirNames[:] = [d for d in DirNames if d not in Args.Exclude]
 FileNames[:] = [f for f in FileNames if f not in Args.Exclude]
+Continue = False
+for Path in Args.Exclude:
+if not os.path.isdir(Path) and not os.path.isfile(Path):
+Path = os.path.join(ExcludeDir, Path)
+if os.path.isdir(Path) and Path.endswith(DirPath):
+DirNames[:] = []
+Continue = True
+elif os.path.isfile(Path):
+FilePaths = FileNames
+for ItemPath in FilePaths:
+FilePath = os.path.join(DirPath, ItemPath)
+if Path.endswith(FilePath):
+FileNames.remove(ItemPath)
+if Continue:
+continue
 for FileName in [f for f in FileNames if any(f.endswith(ext) for ext 
in ExtList)]:
 FileList.append(os.path.join(DirPath, FileName))
 for File in FileList:
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#42981): https://edk2.groups.io/g/devel/message/42981
Mute This Topic: https://groups.io/mt/32239581/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [edk2-platform patch V6] Platform/Intel:Add UniTool into edk2-platforms/Platform/Intel/Tools

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855

UniTool is one python script to generate UQI (Universal Question
Identifier) unicode string for HII question PROMPT string. UQI
string can be used to identify each HII question.
The scripts function will sync up UQI definitions with uni files
based on vfi/vfr/hfr/sd/sdi in the tree.

This script can be run in both Py2 and Py3.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Ard Biesheuvel 
Cc: Leif Lindholm 
Cc: Michael D Kinney 
Signed-off-by: Zhiju.Fan 
---
Updata FileHeader in script

 Platform/Intel/Tools/UniTool/README.txt |  41 +++
 Platform/Intel/Tools/UniTool/UniTool.py | 488 
 2 files changed, 529 insertions(+)
 create mode 100644 Platform/Intel/Tools/UniTool/README.txt
 create mode 100644 Platform/Intel/Tools/UniTool/UniTool.py

diff --git a/Platform/Intel/Tools/UniTool/README.txt 
b/Platform/Intel/Tools/UniTool/README.txt
new file mode 100644
index 00..69da4aca24
--- /dev/null
+++ b/Platform/Intel/Tools/UniTool/README.txt
@@ -0,0 +1,41 @@
+
+How to use UniTool
+-
+The usage of the tool is:
+UniTool.py [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1'] [-d 
'rootDirectory2'] [-d 'rootDirectory3']... [-q e|w]
+   'rootDirectory0' 'uqiFile'|'uqiFileDirectory' 
['excludedDirectory1'] ['excludedDirectory2'] ['excludedDirectory3']...
+
+Function will sync up UQI definitions with uni files based on 
vfi/vfr/hfr/sd/sdi in the tree.
+
+Required Arguments:
+  'rootdirectory0'   path to root directory
+  'uqiFileDirectory' path to UQI file(UqiList.uni)
+  'uqiFile'  UQI file
+
+Options:
+  -hShow this help
+  -bBuild option returns error if any new UQI needs 
assigning
+based on vfi/vfr/hfr/sd/sdi when no -u option is 
specified
+  -uCreate new UQIs that does not already exist in uqiFile 
for
+any string requiring a UQI based on vfi/vfr/hfr/sd/sdi
+NOTE: 'uqiFile' cannot be readonly!
+  -lLanguage deletion option (keeps only English and uqi)
+moves all UQIs to 'uqiFile'
+NOTE: Uni files cannot be readonly!
+  -xExclude 'rootDirectory'/'excludedDirectory1' &
+'rootDirectory'/'excludedDirectory2'... from UQI list 
build
+NOTE: Cannot be the same as rootDirectory
+  -dAdd multiple root directories to process
+  -qPrint warning(w) or return error(e) if different HII 
questions
+are referring same string token
+
+Return error if any duplicated UQI string or value in UQI list or if no 
definition
+for any string referred by HII question when -b or -u is specified
+
+NOTE: Options must be specified before parameters
+
+Notice
+-
+- "S" style will be used if uqiFile needs to be new created.
+  Use the same uqi style if uqiFile is existed. For example,
+  if the exist UqiFile use "\x" style, "\x" will be used.
diff --git a/Platform/Intel/Tools/UniTool/UniTool.py 
b/Platform/Intel/Tools/UniTool/UniTool.py
new file mode 100644
index 00..9b51827cdd
--- /dev/null
+++ b/Platform/Intel/Tools/UniTool/UniTool.py
@@ -0,0 +1,488 @@
+## @file
+# generate UQI (Universal Question Identifier) unicode string for HII question 
PROMPT string. UQI string can be used to
+# identify each HII question.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+import re
+import sys
+import os
+import getopt
+import codecs
+import fnmatch
+import logging
+import argparse
+
+# global variable declarations
+QuestionError = False
+FileHeader = '//\r\n//  FILE auto-generated by UniTool\r\n//\r\n\r\n#langdef 
uqi "uqi"\r\n\r\n'
+UqiList = re.compile('^#string[ \t]+([A-Z_0-9]+)[ \t]+#language[ \t]+uqi[ 
\t\r\n]+"(?:[x\S]{1,2})([0-9a-fA-F]{4,5})"',
+ re.M).findall
+AllUqis = {}
+StringDict = {}
+GlobalVarId = {}
+Options = {}
+
+# Version message
+__prog__ = 'UniTool'
+__description__ = 'The script generate UQI unicode string for HII question 
PROMPT string.\n'
+__copyright__ = 'Copyright (c) 2019, Intel Corporation. All rights 
reserved.'
+__version__ = '%s Version %s' % (__prog__, '0.1 ')
+_Usage = "Syntax:  %s [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1'] [-d 
'rootDirectory2'] [-d 'rootDirectory3']... \n[-q e|w]" \
+ "'rootDirectory0' 'uqiFile'|'uqiFileDirectory' ['excludedDirectory1'] 
['excludedDirectory2'] ['excludedDirectory3']...\n" \
+  % (os.path.basename(sys.argv[0]))
+
+# **
+# description: Get uni file encoding
+#
+# arguments:   Filename - name of uni file
+#
+# returns: utf-8 or utf-16
+#
+def GetUniFileEncoding(Filename):
+#
+# Detect Byte Order Mark at beginning of 

[edk2-devel] [PATCH] BaseTools:Updata the Judgment Method of "--exclude"

[Fan, ZhijuX]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1944

--exclude cannot be used under certain circumstances
1.The value of the parameter USES an absolute path
2.The value of Exclude parameters is based on the value 
  of the Path parameter
Neither of these approaches currently works

This patch is going to fix that issue.

Cc: Liming Gao 
Cc: Bob Feng 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Scripts/FormatDosFiles.py | 18 +++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/BaseTools/Scripts/FormatDosFiles.py 
b/BaseTools/Scripts/FormatDosFiles.py
index d9a880f299..7a5af209f2 100644
--- a/BaseTools/Scripts/FormatDosFiles.py
+++ b/BaseTools/Scripts/FormatDosFiles.py
@@ -20,7 +20,7 @@ import copy
 
 __prog__= 'FormatDosFiles'
 __version__ = '%s Version %s' % (__prog__, '0.10 ')
-__copyright__   = 'Copyright (c) 2018, Intel Corporation. All rights reserved.'
+__copyright__   = 'Copyright (c) 2019, Intel Corporation. All rights reserved.'
 __description__ = 'Convert source files to meet the EDKII C Coding Standards 
Specification.\n'
 DEFAULT_EXT_LIST = ['.h', '.c', '.nasm', '.nasmb', '.asm', '.S', '.inf', 
'.dec', '.dsc', '.fdf', '.uni', '.asl', '.aslc', '.vfr', '.idf', '.txt', 
'.bat', '.py']
 
@@ -46,10 +46,22 @@ def FormatFile(FilePath, Args):
 def FormatFilesInDir(DirPath, ExtList, Args):
 
 FileList = []
+ExcludeDir = DirPath
 for DirPath, DirNames, FileNames in os.walk(DirPath):
 if Args.Exclude:
-DirNames[:] = [d for d in DirNames if d not in Args.Exclude]
-FileNames[:] = [f for f in FileNames if f not in Args.Exclude]
+Continue = False
+for Path in Args.Exclude:
+if not os.path.isdir(Path) and not os.path.isfile(Path):
+Path = os.path.join(ExcludeDir, Path)
+if os.path.isdir(Path) and Path.endswith(DirPath):
+Continue = True
+elif os.path.isfile(Path):
+for ItemPath in FileNames[:]:
+FilePath = os.path.join(DirPath, ItemPath)
+if Path.endswith(FilePath):
+FileNames.remove(ItemPath)
+if Continue:
+continue
 for FileName in [f for f in FileNames if any(f.endswith(ext) for ext 
in ExtList)]:
 FileList.append(os.path.join(DirPath, FileName))
 for File in FileList:
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#42915): https://edk2.groups.io/g/devel/message/42915
Mute This Topic: https://groups.io/mt/32227595/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [edk2-platform patch V5] Platform/Intel:Add UniTool into edk2-platforms/Platform/Intel/Tools

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855

UniTool is one python script to generate UQI (Universal Question
Identifier) unicode string for HII question PROMPT string.
UQI string can be used to identify each HII question.

The scripts function will sync up UQI definitions with uni files
based on vfi/vfr/hfr/sd/sdi in the tree.

This script can be run in both Py2 and Py3.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Ard Biesheuvel 
Cc: Leif Lindholm 
Cc: Michael D Kinney 
Signed-off-by: Zhiju.Fan 
---
Add FileHeader to script

 Platform/Intel/Tools/UniTool/README.txt |  41 +++
 Platform/Intel/Tools/UniTool/UniTool.py | 488 
 2 files changed, 529 insertions(+)
 create mode 100644 Platform/Intel/Tools/UniTool/README.txt
 create mode 100644 Platform/Intel/Tools/UniTool/UniTool.py

diff --git a/Platform/Intel/Tools/UniTool/README.txt 
b/Platform/Intel/Tools/UniTool/README.txt
new file mode 100644
index 00..69da4aca24
--- /dev/null
+++ b/Platform/Intel/Tools/UniTool/README.txt
@@ -0,0 +1,41 @@
+
+How to use UniTool
+-
+The usage of the tool is:
+UniTool.py [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1'] [-d 
'rootDirectory2'] [-d 'rootDirectory3']... [-q e|w]
+   'rootDirectory0' 'uqiFile'|'uqiFileDirectory' 
['excludedDirectory1'] ['excludedDirectory2'] ['excludedDirectory3']...
+
+Function will sync up UQI definitions with uni files based on 
vfi/vfr/hfr/sd/sdi in the tree.
+
+Required Arguments:
+  'rootdirectory0'   path to root directory
+  'uqiFileDirectory' path to UQI file(UqiList.uni)
+  'uqiFile'  UQI file
+
+Options:
+  -hShow this help
+  -bBuild option returns error if any new UQI needs 
assigning
+based on vfi/vfr/hfr/sd/sdi when no -u option is 
specified
+  -uCreate new UQIs that does not already exist in uqiFile 
for
+any string requiring a UQI based on vfi/vfr/hfr/sd/sdi
+NOTE: 'uqiFile' cannot be readonly!
+  -lLanguage deletion option (keeps only English and uqi)
+moves all UQIs to 'uqiFile'
+NOTE: Uni files cannot be readonly!
+  -xExclude 'rootDirectory'/'excludedDirectory1' &
+'rootDirectory'/'excludedDirectory2'... from UQI list 
build
+NOTE: Cannot be the same as rootDirectory
+  -dAdd multiple root directories to process
+  -qPrint warning(w) or return error(e) if different HII 
questions
+are referring same string token
+
+Return error if any duplicated UQI string or value in UQI list or if no 
definition
+for any string referred by HII question when -b or -u is specified
+
+NOTE: Options must be specified before parameters
+
+Notice
+-
+- "S" style will be used if uqiFile needs to be new created.
+  Use the same uqi style if uqiFile is existed. For example,
+  if the exist UqiFile use "\x" style, "\x" will be used.
diff --git a/Platform/Intel/Tools/UniTool/UniTool.py 
b/Platform/Intel/Tools/UniTool/UniTool.py
new file mode 100644
index 00..27fe13cbc1
--- /dev/null
+++ b/Platform/Intel/Tools/UniTool/UniTool.py
@@ -0,0 +1,488 @@
+## @file
+# generate UQI (Universal Question Identifier) unicode string for HII question 
PROMPT string. UQI string can be used to
+# identify each HII question.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+import re
+import sys
+import os
+import getopt
+import codecs
+import fnmatch
+import logging
+import argparse
+
+# global variable declarations
+QuestionError = False
+FileHeader = '//\n//  FILE auto-generated by UniTool\n//\n\n#langdef uqi 
"uqi"\n\n'
+UqiList = re.compile('^#string[ \t]+([A-Z_0-9]+)[ \t]+#language[ \t]+uqi[ 
\t\r\n]+"(?:[x\S]{1,2})([0-9a-fA-F]{4,5})"',
+ re.M).findall
+AllUqis = {}
+StringDict = {}
+GlobalVarId = {}
+Options = {}
+
+# Version message
+__prog__ = 'UniTool'
+__description__ = 'The script generate UQI unicode string for HII question 
PROMPT string.\n'
+__copyright__ = 'Copyright (c) 2019, Intel Corporation. All rights 
reserved.'
+__version__ = '%s Version %s' % (__prog__, '0.1 ')
+_Usage = "Syntax:  %s [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1'] [-d 
'rootDirectory2'] [-d 'rootDirectory3']... \n[-q e|w]" \
+ "'rootDirectory0' 'uqiFile'|'uqiFileDirectory' ['excludedDirectory1'] 
['excludedDirectory2'] ['excludedDirectory3']...\n" \
+  % (os.path.basename(sys.argv[0]))
+
+# **
+# description: Get uni file encoding
+#
+# arguments:   Filename - name of uni file
+#
+# returns: utf-8 or utf-16
+#
+def GetUniFileEncoding(Filename):
+#
+# Detect Byte Order Mark at beginning of file.  Default 

[edk2-devel] [PATCH V2] BaseTools:Linux changes the way the latest version is judged

[Fan, ZhijuX]

Some Linux servers do not have BC installed,so errors occur.
So the judgment was changed to avoid this error.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
Change "\<" to "<" and add as two "[]"

 edksetup.sh | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/edksetup.sh b/edksetup.sh
index c7b2e1e201..0723e7b85a 100755
--- a/edksetup.sh
+++ b/edksetup.sh
@@ -122,8 +122,7 @@ function SetupPython3()
 export PYTHON_COMMAND=$python
 continue
   fi
-  ret=`echo "$origin_version < $python_version" |bc`
-  if [ "$ret" -eq 1 ]; then
+  if [[ "$origin_version" < "$python_version" ]]; then
 origin_version=$python_version
 export PYTHON_COMMAND=$python
   fi
@@ -165,8 +164,7 @@ function SetupPython()
 export PYTHON_COMMAND=$python
 continue
   fi
-  ret=`echo "$origin_version < $python_version" |bc`
-  if [ "$ret" -eq 1 ]; then
+  if [[ "$origin_version" < "$python_version" ]]; then
 origin_version=$python_version
 export PYTHON_COMMAND=$python
   fi
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#42864): https://edk2.groups.io/g/devel/message/42864
Mute This Topic: https://groups.io/mt/32211945/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [edk2-platform patch V3] Platform/Intel:Add UniTool into edk2-platforms/Platform/Intel/Tools

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855

UniTool is one python script to generate UQI (Universal Question
Identifier) unicode string for HII question PROMPT string. UQI
string can be used to identify each HII question.
The scripts function will sync up UQI definitions with uni files
based on vfi/vfr/hfr/sd/sdi in the tree.

This script can be run in both Py2 and Py3.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Ard Biesheuvel 
Cc: Leif Lindholm 
Cc: Michael D Kinney 
Signed-off-by: Zhiju.Fan 
---
Add README.txt to scripts

 Platform/Intel/Tools/UniTool/README.txt |  46 +++
 Platform/Intel/Tools/UniTool/UniTool.py | 499 
 2 files changed, 545 insertions(+)
 create mode 100644 Platform/Intel/Tools/UniTool/README.txt
 create mode 100644 Platform/Intel/Tools/UniTool/UniTool.py

diff --git a/Platform/Intel/Tools/UniTool/README.txt 
b/Platform/Intel/Tools/UniTool/README.txt
new file mode 100644
index 00..3e59a6c75c
--- /dev/null
+++ b/Platform/Intel/Tools/UniTool/README.txt
@@ -0,0 +1,46 @@
+To update the UniTool.exe utility
+-
+- Must have python and cx_freeze installed with the default location.
+- After modify the UniTool.py script as desired, launch the runit.cmd script
+- New version generated in UniTool.exe
+
+How to use UniTool
+-
+The usage of the tool is:
+UniTool.py [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1'] [-d 
'rootDirectory2'] [-d 'rootDirectory3']... [-q e|w]
+   'rootDirectory0' 'uqiFile'|'uqiFileDirectory' 
['excludedDirectory1'] ['excludedDirectory2'] ['excludedDirectory3']...
+
+Function will sync up UQI definitions with uni files based on 
vfi/vfr/hfr/sd/sdi in the tree.
+
+Required Arguments:
+  'rootdirectory0'   path to root directory
+  'uqiFileDirectory' path to UQI file(UqiList.uni)
+  'uqiFile'  UQI file
+
+Options:
+  -hShow this help
+  -bBuild option returns error if any new UQI needs 
assigning
+based on vfi/vfr/hfr/sd/sdi when no -u option is 
specified
+  -uCreate new UQIs that does not already exist in uqiFile 
for
+any string requiring a UQI based on vfi/vfr/hfr/sd/sdi
+NOTE: 'uqiFile' cannot be readonly!
+  -lLanguage deletion option (keeps only English and uqi)
+moves all UQIs to 'uqiFile'
+NOTE: Uni files cannot be readonly!
+  -xExclude 'rootDirectory'/'excludedDirectory1' &
+'rootDirectory'/'excludedDirectory2'... from UQI list 
build
+NOTE: Cannot be the same as rootDirectory
+  -dAdd multiple root directories to process
+  -qPrint warning(w) or return error(e) if different HII 
questions
+are referring same string token
+
+Return error if any duplicated UQI string or value in UQI list or if no 
definition
+for any string referred by HII question when -b or -u is specified
+
+NOTE: Options must be specified before parameters
+
+Notice
+-
+- "S" style will be used if uqiFile needs to be new created.
+  Use the same uqi style if uqiFile is existed. For example,
+  if the exist UqiFile use "\x" style, "\x" will be used.
diff --git a/Platform/Intel/Tools/UniTool/UniTool.py 
b/Platform/Intel/Tools/UniTool/UniTool.py
new file mode 100644
index 00..df2b1036ce
--- /dev/null
+++ b/Platform/Intel/Tools/UniTool/UniTool.py
@@ -0,0 +1,499 @@
+## @file
+# generate UQI (Universal Question Identifier) unicode string for HII question 
PROMPT string. UQI string can be used to
+# identify each HII question.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+import re
+import sys
+import os
+import getopt
+import codecs
+import fnmatch
+import logging
+import argparse
+
+# global variable declarations
+QuestionError = False
+UqiList = re.compile('^#string[ \t]+([A-Z_0-9]+)[ \t]+#language[ \t]+uqi[ 
\t\r\n]+"(?:[x\S]{1,2})([0-9a-fA-F]{4,5})"',
+ re.M).findall
+AllUqis = {}
+StringDict = {}
+GlobalVarId = {}
+Options = {}
+
+# Version message
+__prog__ = 'UniTool'
+__description__ = 'generate UQI unicode string for HII question PROMPT string.'
+__copyright__ = 'Copyright (c) 2019, Intel Corporation. All rights 
reserved.'
+__version__ = '%s Version %s' % (__prog__, '0.1 ')
+_Usage = "Syntax:  %s [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1'] [-d 
'rootDirectory2'] [-d 'rootDirectory3']... \n[-q e|w]" \
+ "'rootDirectory0' 'uqiFile'|'uqiFileDirectory' ['excludedDirectory1'] 
['excludedDirectory2'] ['excludedDirectory3']...\n" \
+  % (os.path.basename(sys.argv[0]))
+_Epilog = """\n\nFunction will sync up UQI definitions with uni files based on 
vfi/vfr/hfr/sd/sdi in the tree.\n
+ Required Arguments:
+   

FW: [edk2-devel] [PATCH] BaseTools:Linux changes the way the latest version is judged

[Fan, ZhijuX]

Using "-lt" will report the following errors, but "\<" will not
3.6: integer expression expected
3.7: integer expression expected

if [[ "$origin_version" < "$python_version" ]];then
It is equivalent to
if [ "$origin_version" \< "$python_version" ];then
These two approaches work

Any question, please let me know. Thanks.

Best Regards
Fan Zhiju



-Original Message-
From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of 
rebe...@bluestop.org
Sent: Monday, June 24, 2019 11:25 AM
To: devel@edk2.groups.io; Fan, ZhijuX 
Cc: Gao, Liming ; Feng, Bob C 
Subject: Re: [edk2-devel] [PATCH] BaseTools:Linux changes the way the latest 
version is judged

On 2019-06-20 20:33, Fan, ZhijuX wrote:
> -  ret=`echo "$origin_version < $python_version" |bc`
> -  if [ "$ret" -eq 1 ]; then
> +  if [ "$origin_version" \< "$python_version" ]; then
>  origin_version=$python_version
>  export PYTHON_COMMAND=$python
>fi


I think using the "-lt" operator might be better (more standard). e.g.:


if [ "$origin_version" -lt "$python_version" ]; then



-- 
Rebecca Cran





-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#42774): https://edk2.groups.io/g/devel/message/42774
Mute This Topic: https://groups.io/mt/32199579/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [edk2-platform patch V2] Platform/Intel:Add UniTool into edk2-platforms/Platform/Intel/Tools

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855

UniTool is one python script to generate UQI (Universal Question
Identifier) unicode string for HII question PROMPT string. UQI
string can be used to identify each HII question.
The scripts function will sync up UQI definitions with uni files
based on vfi/vfr/hfr/sd/sdi in the tree.

This script can be run in both Py2 and Py3.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Ard Biesheuvel 
Cc: Leif Lindholm 
Cc: Michael D Kinney 
Signed-off-by: Zhiju.Fan 
---
V2:add argparse to scripts

 Platform/Intel/Tools/UniTool/UniTool.py | 494 
 1 file changed, 494 insertions(+)
 create mode 100644 Platform/Intel/Tools/UniTool/UniTool.py

diff --git a/Platform/Intel/Tools/UniTool/UniTool.py 
b/Platform/Intel/Tools/UniTool/UniTool.py
new file mode 100644
index 00..298ea47909
--- /dev/null
+++ b/Platform/Intel/Tools/UniTool/UniTool.py
@@ -0,0 +1,494 @@
+## @file
+# generate UQI (Universal Question Identifier) unicode string for HII question 
PROMPT string. UQI string can be used to
+# identify each HII question.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+import re
+import sys
+import os
+import getopt
+import codecs
+import fnmatch
+import logging
+import argparse
+
+# global variable declarations
+QuestionError = False
+UqiList = re.compile('^#string[ \t]+([A-Z_0-9]+)[ \t]+#language[ \t]+uqi[ 
\t\r\n]+"(?:[x\S]{1,2})([0-9a-fA-F]{4,5})"',
+ re.M).findall
+AllUqis = {}
+StringDict = {}
+GlobalVarId = {}
+Options = {}
+
+# Version message
+__prog__ = 'UniTool'
+__description__ = 'generate UQI unicode string for HII question PROMPT string.'
+__copyright__ = 'Copyright (c) 2019, Intel Corporation. All rights 
reserved. '
+__version__ = '%s Version %s' % (__prog__, '0.1 ')
+_Usage = "Syntax:  %s [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1'] [-d 
'rootDirectory2'] [-d 'rootDirectory3']... \n[-q e|w]" \
+ "'rootDirectory0' 'uqiFile'|'uqiFileDirectory' ['excludedDirectory1'] 
['excludedDirectory2'] ['excludedDirectory3']...\n" \
+ """\nFunction will sync up UQI definitions with uni files based on 
vfi/vfr/hfr/sd/sdi in the tree.\n
+ Required Arguments:
+   'rootdirectory0'   path to root directory
+   'uqiFileDirectory' path to UQI file(UqiList.uni)
+   'uqiFile'  UQI file
+
+ Return error if any duplicated UQI string or value in UQI list or if 
no definition
+ for any string referred by HII question when -b or -u is specified
+
+ NOTE: Options must be specified before parameters
+ """ % (os.path.basename(sys.argv[0]))
+
+
+# **
+# description: Get uni file encoding
+#
+# arguments:   Filename - name of uni file
+#
+# returns: utf-8 or utf-16
+#
+def GetUniFileEncoding(Filename):
+#
+# Detect Byte Order Mark at beginning of file.  Default to UTF-8
+#
+Encoding = 'utf-8'
+
+#
+# Read file
+#
+try:
+with open(Filename, mode='rb') as UniFile:
+FileIn = UniFile.read()
+except:
+return Encoding
+
+if (FileIn.startswith(codecs.BOM_UTF16_BE) or 
FileIn.startswith(codecs.BOM_UTF16_LE)):
+Encoding = 'utf-16'
+
+return Encoding
+
+
+# rewrite function os.path.walk
+def Walk(Top, Func, Arg):
+try:
+Names = os.listdir(Top)
+except os.error:
+return
+Func(Arg, Top, Names)
+for Name in Names:
+Name = os.path.join(Top, Name)
+if os.path.isdir(Name):
+Walk(Name, Func, Arg)
+
+
+# **
+# description: Parses commandline arguments and options
+#  Calls function processUni to build dictionary of strings
+#  Calls other functions according to user specified options
+#
+# arguments:   argv - contains all input from command line
+#   - must contain path to root directory
+#   - may contain options -h, -u, -l, -b or -x before path
+#
+# returns: none
+#
+def main():
+# Read input arguments and options
+global AllUqis, UqiList, QuestionError
+parser = argparse.ArgumentParser(prog=__prog__,
+ description=__description__ + 
__copyright__,
+ conflict_handler='resolve')
+parser.add_argument('-v', '--version', action='version', 
version=__version__,
+help="show program's version number and exit")
+parser.add_argument('-b', '--build', action='store_true', 
dest='BuildOption',
+help="Build option returns error if any new UQI needs 
assigning " \
+ "based on vfi/vfr/hfr/sd/sdi when no -u option is 
specified")
+parser.add_argument('-u', '--updata', action='store_true', 

[edk2-devel] [PATCH V4] BaseTools:Add DetectNotUsedItem.py to Edk2\BaseTools\Scripts

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1850

This script is used to Detect unreferenced PCD and GUID/Protocols/PPIs.
The input parameters are Dec file and package directory.

This script can be run in both Py2 and Py3.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Scripts/DetectNotUsedItem.py | 198 +
 1 file changed, 198 insertions(+)
 create mode 100644 BaseTools/Scripts/DetectNotUsedItem.py

diff --git a/BaseTools/Scripts/DetectNotUsedItem.py 
b/BaseTools/Scripts/DetectNotUsedItem.py
new file mode 100644
index 00..7e3568fcf9
--- /dev/null
+++ b/BaseTools/Scripts/DetectNotUsedItem.py
@@ -0,0 +1,198 @@
+## @file
+# Detect unreferenced PCD and GUID/Protocols/PPIs.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+'''
+DetectNotUsedItem
+'''
+import re
+import os
+import sys
+import argparse
+
+#
+# Globals for help information
+#
+__prog__ = 'DetectNotUsedItem'
+__version__ = '%s Version %s' % (__prog__, '0.1')
+__copyright__ = 'Copyright (c) 2019, Intel Corporation. All rights reserved.'
+__description__ = "Detect unreferenced PCD and GUID/Protocols/PPIs.\n"
+
+SectionList = ["LibraryClasses", "Guids", "Ppis", "Protocols", "Pcd"]
+
+
+class PROCESS(object):
+
+def __init__(self, DecPath, InfDirs):
+self.Dec = DecPath
+self.InfPath = InfDirs
+self.Log = []
+
+def ParserDscFdfInfFile(self):
+AllContentList = []
+for File in self.SearchbyExt([".dsc", ".fdf", ".inf"]):
+AllContentList += self.ParseDscFdfInfContent(File)
+return AllContentList
+
+# Search File by extension name
+def SearchbyExt(self, ExtList):
+FileList = []
+for path in self.InfPath:
+if type(ExtList) == type(''):
+for root, _, files in os.walk(path, topdown=True, 
followlinks=False):
+for filename in files:
+if filename.endswith(ExtList):
+FileList.append(os.path.join(root, filename))
+elif type(ExtList) == type([]):
+for root, _, files in os.walk(path, topdown=True, 
followlinks=False):
+for filename in files:
+for Ext in ExtList:
+if filename.endswith(Ext):
+FileList.append(os.path.join(root, filename))
+return FileList
+
+# Parse DEC file to get Line number and Name
+# return section name, the Item Name and comments line number
+def ParseDecContent(self):
+SectionRE = re.compile(r'\[(.*)\]')
+Flag = False
+Comments = {}
+Comment_Line = []
+ItemName = {}
+with open(self.Dec, 'r') as F:
+for Index, content in enumerate(F):
+NotComment = not content.strip().startswith("#")
+Section = SectionRE.findall(content)
+if Section and NotComment:
+Flag = self.IsNeedParseSection(Section[0])
+if Flag:
+Comment_Line.append(Index)
+if NotComment:
+if content != "\n" and content != "\r\n":
+ItemName[Index] = 
content.split('=')[0].split('|')[0].split('#')[0].strip()
+Comments[Index] = Comment_Line
+Comment_Line = []
+return ItemName, Comments
+
+def IsNeedParseSection(self, SectionName):
+for item in SectionList:
+if item in SectionName:
+return True
+return False
+
+# Parse DSC, FDF, INF File, remove comments, return Lines list
+def ParseDscFdfInfContent(self, File):
+with open(File, 'r') as F:
+lines = F.readlines()
+for Index in range(len(lines) - 1, -1, -1):
+if lines[Index].strip().startswith("#") or lines[Index] == "\n" or 
lines[Index] == "\r\n":
+lines.remove(lines[Index])
+elif "#" in lines[Index]:
+lines[Index] = lines[Index].split("#")[0].strip()
+else:
+lines[Index] = lines[Index].strip()
+return lines
+
+def DetectNotUsedItem(self):
+NotUsedItem = {}
+DecItem, DecComments = self.ParseDecContent()
+InfDscFdfContent = self.ParserDscFdfInfFile()
+for LineNum in list(DecItem.keys()):
+DecItemName = DecItem[LineNum]
+Match_reg = re.compile("(?

[edk2-devel] [edk2-platform patch 1/2 V2] Platform/Intel:Add GenBiosId into edk2-platforms/Platform/Intel/Tools

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855

GenBiosId is a tool to generate the BIOS ID binary file which uses
the data from the configuration file.
This tool can be run in both Py2 and Py3.

This patch is going to add GenBiosId to Platform/Intel/Tools

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Ard Biesheuvel 
Cc: Leif Lindholm 
Cc: Michael D Kinney 
Signed-off-by: Zhiju.Fan 
---
 Platform/Intel/Tools/GenBiosId/BiosId.env   |  27 +
 Platform/Intel/Tools/GenBiosId/GenBiosId.py | 180 
 2 files changed, 207 insertions(+)
 create mode 100644 Platform/Intel/Tools/GenBiosId/BiosId.env
 create mode 100644 Platform/Intel/Tools/GenBiosId/GenBiosId.py

diff --git a/Platform/Intel/Tools/GenBiosId/BiosId.env 
b/Platform/Intel/Tools/GenBiosId/BiosId.env
new file mode 100644
index 00..e1e913da76
--- /dev/null
+++ b/Platform/Intel/Tools/GenBiosId/BiosId.env
@@ -0,0 +1,27 @@
+## @file
+#  This file is used to define the BIOS ID parameters of the build.
+#  This file is processed by GenBiosId.
+#  Here, it is just a template and can be customized by user.
+#
+#  BIOS ID string format:
+#
$(BOARD_ID)$(BOARD_REV).$(BOARD_EXT).$(VERSION_MAJOR).$(BUILD_TYPE)$(VERSION_MINOR).YYMMDDHHMM
+#  All fields must have a fixed length. YYMMDDHHMM is UTC time.
+#Example: "EMLATOR1.000.0001.D01.1906141517"
+#
+#  If DATE is specified for YYMMDD and TIME is specified for HHMM like below,
+#  GenBiosId will use the value of DATE and TIME to fill YYMMDDHHMM,
+#  otherwise GenBiosId will fill YYMMDDHHMM with current UTC time of the build 
machine.
+#DATE  = 190614
+#TIME  = 1517
+#
+#  Copyright (c) 2019, Intel Corporation. All rights reserved.
+#  SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+##
+[config]
+BOARD_ID  = KBLRVP3
+BOARD_REV = 1
+BOARD_EXT = 000
+BUILD_TYPE= D
+VERSION_MAJOR = 0001
+VERSION_MINOR = 01
diff --git a/Platform/Intel/Tools/GenBiosId/GenBiosId.py 
b/Platform/Intel/Tools/GenBiosId/GenBiosId.py
new file mode 100644
index 00..20fb7592b4
--- /dev/null
+++ b/Platform/Intel/Tools/GenBiosId/GenBiosId.py
@@ -0,0 +1,180 @@
+## @file
+# Trim files preprocessed by compiler
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+##
+# Import Modules
+#
+import os
+import sys
+import time
+import logging
+import struct
+import datetime
+import argparse
+import platform
+
+try:
+from configparser import ConfigParser
+except:
+from ConfigParser import ConfigParser
+
+# Config message
+_BIOS_Signature = "$IBIOSI$"
+_SectionKeyName = '__name__'
+_SectionName = 'config'
+
+_ConfigItem = {
+"BOARD_ID": {'Value': '', 'Length': 7},
+"BOARD_REV": {'Value': '', 'Length': 1},
+"BOARD_EXT": {'Value': '', 'Length': 3},
+"BUILD_TYPE": {'Value': '', 'Length': 1},
+"VERSION_MAJOR": {'Value': '', 'Length': 4},
+"VERSION_MINOR": {'Value': '00', 'Length': 2},
+}
+
+# Version message
+__prog__ = 'GenBiosld'
+__description__ = 'Trim files preprocessed by compiler'
+__copyright__ = 'Copyright (c) 2019, Intel Corporation. All rights 
reserved. '
+__version__ = '%s Version %s' % (__prog__, '0.1 ')
+
+# ExtraData message
+_Usage = "Usage: GenBiosId -i Configfile -o OutputFile [-ot OutputTextFile]"
+_ConfigSectionNotDefine = "Not support the config file format, need config 
section"
+_ErrorMessageTemplate = '\n\n%(tool)s...\n : error: %(msg)s\n\t%(extra)s'
+_ErrorLogger = logging.getLogger("tool_error")
+_ErrorFormatter = logging.Formatter("%(message)s")
+_ConfigLenInvalid = "Config item %s length is invalid"
+_ConfigItemInvalid = "Item %s is invalid"
+
+# Error message
+INFO = 20
+ERRORCODE = 50
+OPTION_MISSING = 'Missing option'
+FORMAT_INVALID = 'Invalid syntax/format'
+FILE_NOT_FOUND = 'File/directory not found in workspace'
+FORMAT_UNKNOWN_ERROR = 'Unknown error in syntax/format'
+FORMAT_NOT_SUPPORTED = 'Not supported syntax/format'
+
+
+def SetEdkLogger():
+_ErrorLogger.setLevel(INFO)
+_ErrorCh = logging.StreamHandler(sys.stderr)
+_ErrorCh.setFormatter(_ErrorFormatter)
+_ErrorLogger.addHandler(_ErrorCh)
+return _ErrorLogger
+
+
+# Output the error message and exit the tool
+def EdkLogger(ToolName, Message, ExtraData):
+_ErrorLogger = SetEdkLogger()
+TemplateDict = {"tool": ToolName, "msg": Message, "extra": ExtraData}
+LogText = _ErrorMessageTemplate % TemplateDict
+_ErrorLogger.log(ERRORCODE, LogText)
+sys.exit(1)
+
+
+# Open the file in the correct way
+def FileOpen(FileName, Mode, Buffer=-1):
+def LongFilePath(FileName):
+FileName = os.path.normpath(FileName)
+if platform.system() == 'Windows':
+if FileName.startswith('?\\'):
+return FileName
+if FileName.startswith(''):
+return '?\\UNC\\' + FileName[2:]
+if os.path.isabs(FileName):
+return '?\\' + FileName
+return FileName
+
+

Re: [edk2-devel] [edk2-platform patch 2/2] Platform/Intel:Add UniTool into edk2-platforms/Platform/Intel/Tools

[Fan, ZhijuX]

There are several reasons why argparse is not used:

1. The logic and parameters of this script are simple.
2. This script is ported from another directory, Only functional changes have 
been made to it, 
 and its architecture is basically unchanged,
3. Printing a Help Message when an error occurs is an original feature of the 
script, 
 so it is not modified for argparse.



Any question, please let me know. Thanks.

Best Regards
Fan Zhiju




> -Original Message-
> From: Leif Lindholm [mailto:leif.lindh...@linaro.org]
> Sent: Friday, June 21, 2019 5:04 PM
> To: Fan, ZhijuX 
> Cc: devel@edk2.groups.io; Gao, Liming ; Feng, Bob C
> ; Ard Biesheuvel ;
> Kinney, Michael D 
> Subject: Re: [edk2-platform patch 2/2] Platform/Intel:Add UniTool into edk2-
> platforms/Platform/Intel/Tools
> 
> On Fri, Jun 21, 2019 at 01:58:57AM +, Fan, ZhijuX wrote:
> > BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855
> >
> > UniTool is one python script to generate UQI (Universal Question
> > Identifier) unicode string for HII question PROMPT string. UQI string
> > can be used to identify each HII question.
> > The scripts function will sync up UQI definitions with uni files based
> > on vfi/vfr/hfr/sd/sdi in the tree.
> >
> > This script can be run in both Py2 and Py3.
> >
> > Cc: Liming Gao 
> > Cc: Bob Feng 
> > Cc: Ard Biesheuvel 
> > Cc: Leif Lindholm 
> > Cc: Michael D Kinney 
> > Signed-off-by: Zhiju.Fan 
> > ---
> >  Platform/Intel/Tools/UniTool/UniTool.py | 515
> > 
> >  1 file changed, 515 insertions(+)
> >  create mode 100644 Platform/Intel/Tools/UniTool/UniTool.py
> >
> > diff --git a/Platform/Intel/Tools/UniTool/UniTool.py
> > b/Platform/Intel/Tools/UniTool/UniTool.py
> > new file mode 100644
> > index 00..aec25b51c2
> > --- /dev/null
> > +++ b/Platform/Intel/Tools/UniTool/UniTool.py
> > @@ -0,0 +1,515 @@
> > +## @file
> > +# generate UQI (Universal Question Identifier) unicode string for HII
> > +question PROMPT string. UQI string can be used to # identify each HII
> question.
> > +#
> > +# Copyright (c) 2019, Intel Corporation. All rights reserved.
> > +#
> > +# SPDX-License-Identifier: BSD-2-Clause-Patent #
> > +
> > +import re
> > +import sys
> > +import os
> > +import getopt
> > +import codecs
> > +import fnmatch
> > +import logging
> > +
> > +# global variable declarations
> > +QuestionError = False
> > +UqiList = re.compile('^#string[ \t]+([A-Z_0-
> 9]+)[ \t]+#language[ \t]+uqi[ \t\r\n]+"(?:[x\S]{1,2})([0-9a-fA-F]{4,5})"',
> > + re.M).findall
> > +AllUqis = {}
> > +StringDict = {}
> > +GlobalVarId = {}
> > +Options = {}
> > +
> > +
> > +#
> >
> +*
> 
> > +* # description: Prints help information #
> > +# arguments:   none
> > +#
> > +# returns: none
> > +#
> > +
> > +def Usage():
> 
> Why not use argparse?
> 
> /
> Leif
> 
> > +print("Syntax:  %s [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1']
> > +[-d 'rootDirectory2'] [-d 'rootDirectory3']... [-q e|w] \ 'rootDirectory0'
> 'uqiFile'|'uqiFileDirectory' ['excludedDirectory1'] ['excludedDirectory2']
> ['excludedDirectory3']...\n%s" %
> > +  (os.path.basename(sys.argv[0]),
> > +   """\nFunction will sync up UQI definitions with uni files based 
> > on
> vfi/vfr/hfr/sd/sdi in the tree.\n
> > + Required Arguments:
> > +   'rootdirectory0'   path to root directory
> > +   'uqiFileDirectory' path to UQI file(UqiList.uni)
> > +   'uqiFile'  UQI file
> > +
> > + Options:
> > +   -hShow this help
> > +   -bBuild option returns error if any new UQI 
> > needs assigning
> > + based on vfi/vfr/hfr/sd/sdi when no -u option 
> > is specified
> > +   -uCreate new UQIs that does not already exist 
> > in uqiFile for
> > + any string requiring a UQI based on 
> > vfi/vfr/hfr/sd/sdi
> > + NOTE: 'uqiFile' cannot be readonly!
> > +   -lLanguage deletion option (keeps only English 
> > and uqi)
> > + moves all UQIs to 'uqiFile'
> > + NOTE: Uni files cannot be readonly!
> > +   -x 

Re: [edk2-devel] [edk2-platform patch 1/2] Platform/Intel:Add GenBiosId into edk2-platforms/Platform/Intel/Tools

[Fan, ZhijuX]

Hi:

Thank you for your comments. The output 1 was designed as a standard error.
Function EdkLogger ()   sys.exit(1)  "1" missing while writing the script




Any question, please let me know. Thanks.

Best Regards
Fan Zhiju



> -Original Message-
> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
> Leif Lindholm
> Sent: Friday, June 21, 2019 5:01 PM
> To: Fan, ZhijuX 
> Cc: devel@edk2.groups.io; Gao, Liming ; Feng, Bob C
> ; Ard Biesheuvel ;
> Kinney, Michael D 
> Subject: Re: [edk2-devel] [edk2-platform patch 1/2] Platform/Intel:Add
> GenBiosId into edk2-platforms/Platform/Intel/Tools
> 
> On Fri, Jun 21, 2019 at 01:58:17AM +, Fan, ZhijuX wrote:
> > BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855
> >
> > GenBiosId is a tool to generate the BIOS ID binary file which uses the
> > data from the configuration file.
> > This tool can be run in both Py2 and Py3.
> >
> > This patch is going to add GenBiosId to Platform/Intel/Tools
> >
> > Cc: Liming Gao 
> > Cc: Bob Feng 
> > Cc: Ard Biesheuvel 
> > Cc: Leif Lindholm 
> > Cc: Michael D Kinney 
> > Signed-off-by: Zhiju.Fan 
> > ---
> >  Platform/Intel/Tools/GenBiosId/BiosId.env   |  27 +
> >  Platform/Intel/Tools/GenBiosId/GenBiosId.py | 180
> > 
> >  2 files changed, 207 insertions(+)
> >  create mode 100644 Platform/Intel/Tools/GenBiosId/BiosId.env
> >  create mode 100644 Platform/Intel/Tools/GenBiosId/GenBiosId.py
> >
> > diff --git a/Platform/Intel/Tools/GenBiosId/BiosId.env
> > b/Platform/Intel/Tools/GenBiosId/BiosId.env
> > new file mode 100644
> > index 00..e1e913da76
> > --- /dev/null
> > +++ b/Platform/Intel/Tools/GenBiosId/BiosId.env
> > @@ -0,0 +1,27 @@
> > +## @file
> > +#  This file is used to define the BIOS ID parameters of the build.
> > +#  This file is processed by GenBiosId.
> > +#  Here, it is just a template and can be customized by user.
> > +#
> > +#  BIOS ID string format:
> > +#
> $(BOARD_ID)$(BOARD_REV).$(BOARD_EXT).$(VERSION_MAJOR).$(BUILD_T
> YPE)$(VERSION_MINOR).YYMMDDHHMM
> > +#  All fields must have a fixed length. YYMMDDHHMM is UTC time.
> > +#Example: "EMLATOR1.000.0001.D01.1906141517"
> > +#
> > +#  If DATE is specified for YYMMDD and TIME is specified for HHMM
> > +like below, #  GenBiosId will use the value of DATE and TIME to fill
> > +YYMMDDHHMM, #  otherwise GenBiosId will fill YYMMDDHHMM with
> current UTC time of the build machine.
> > +#DATE  = 190614
> > +#TIME  = 1517
> > +#
> > +#  Copyright (c) 2019, Intel Corporation. All rights reserved. #
> > +SPDX-License-Identifier: BSD-2-Clause-Patent # ## [config]
> > +BOARD_ID  = KBLRVP3
> > +BOARD_REV = 1
> > +BOARD_EXT = 000
> > +BUILD_TYPE= D
> > +VERSION_MAJOR = 0001
> > +VERSION_MINOR = 01
> > diff --git a/Platform/Intel/Tools/GenBiosId/GenBiosId.py
> > b/Platform/Intel/Tools/GenBiosId/GenBiosId.py
> > new file mode 100644
> > index 00..20fb7592b4
> > --- /dev/null
> > +++ b/Platform/Intel/Tools/GenBiosId/GenBiosId.py
> > @@ -0,0 +1,180 @@
> > +## @file
> > +# Trim files preprocessed by compiler # # Copyright (c) 2019, Intel
> > +Corporation. All rights reserved. # SPDX-License-Identifier:
> > +BSD-2-Clause-Patent #
> > +
> > +##
> > +# Import Modules
> > +#
> > +import os
> > +import sys
> > +import time
> > +import logging
> > +import struct
> > +import datetime
> > +import argparse
> > +import platform
> > +
> > +try:
> > +from configparser import ConfigParser
> > +except:
> > +from ConfigParser import ConfigParser
> > +
> > +# Config message
> > +_BIOS_Signature = "$IBIOSI$"
> > +_SectionKeyName = '__name__'
> > +_SectionName = 'config'
> > +
> > +_ConfigItem = {
> > +"BOARD_ID": {'Value': '', 'Length': 7},
> > +"BOARD_REV": {'Value': '', 'Length': 1},
> > +"BOARD_EXT": {'Value': '', 'Length': 3},
> > +"BUILD_TYPE": {'Value': '', 'Length': 1},
> > +"VERSION_MAJOR": {'Value': '', 'Length': 4},
> > +"VERSION_MINOR": {'Value': '00', 'Length': 2}, }
> > +
> > +# Version message
> > +__prog__ = 'GenBiosld'
> > +__description__ = 'Trim files preprocessed by compiler'
> > +__copyright__ = 'Copyright (c) 2019, Intel Corporation. All rights
> reserved. '
> > +__version__ = '%s Vers

[edk2-devel] [PATCH] BaseTools:Linux changes the way the latest version is judged

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1929

Some Linux servers do not have BC installed,so errors occur.
So the judgment was changed to avoid this error.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 edksetup.sh | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/edksetup.sh b/edksetup.sh
index c7b2e1e201..0723e7b85a 100755
--- a/edksetup.sh
+++ b/edksetup.sh
@@ -122,8 +122,7 @@ function SetupPython3()
 export PYTHON_COMMAND=$python
 continue
   fi
-  ret=`echo "$origin_version < $python_version" |bc`
-  if [ "$ret" -eq 1 ]; then
+  if [ "$origin_version" \< "$python_version" ]; then
 origin_version=$python_version
 export PYTHON_COMMAND=$python
   fi
@@ -165,8 +164,7 @@ function SetupPython()
 export PYTHON_COMMAND=$python
 continue
   fi
-  ret=`echo "$origin_version < $python_version" |bc`
-  if [ "$ret" -eq 1 ]; then
+  if [ "$origin_version" \< "$python_version" ]; then
 origin_version=$python_version
 export PYTHON_COMMAND=$python
   fi
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#42682): https://edk2.groups.io/g/devel/message/42682
Mute This Topic: https://groups.io/mt/32154778/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [edk2-platform patch 2/2] Platform/Intel:Add UniTool into edk2-platforms/Platform/Intel/Tools

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855

UniTool is one python script to generate UQI (Universal Question
Identifier) unicode string for HII question PROMPT string. UQI
string can be used to identify each HII question.
The scripts function will sync up UQI definitions with uni files
based on vfi/vfr/hfr/sd/sdi in the tree.

This script can be run in both Py2 and Py3.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Ard Biesheuvel 
Cc: Leif Lindholm 
Cc: Michael D Kinney 
Signed-off-by: Zhiju.Fan 
---
 Platform/Intel/Tools/UniTool/UniTool.py | 515 
 1 file changed, 515 insertions(+)
 create mode 100644 Platform/Intel/Tools/UniTool/UniTool.py

diff --git a/Platform/Intel/Tools/UniTool/UniTool.py 
b/Platform/Intel/Tools/UniTool/UniTool.py
new file mode 100644
index 00..aec25b51c2
--- /dev/null
+++ b/Platform/Intel/Tools/UniTool/UniTool.py
@@ -0,0 +1,515 @@
+## @file
+# generate UQI (Universal Question Identifier) unicode string for HII question 
PROMPT string. UQI string can be used to
+# identify each HII question.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+import re
+import sys
+import os
+import getopt
+import codecs
+import fnmatch
+import logging
+
+# global variable declarations
+QuestionError = False
+UqiList = re.compile('^#string[ \t]+([A-Z_0-9]+)[ \t]+#language[ \t]+uqi[ 
\t\r\n]+"(?:[x\S]{1,2})([0-9a-fA-F]{4,5})"',
+ re.M).findall
+AllUqis = {}
+StringDict = {}
+GlobalVarId = {}
+Options = {}
+
+
+# **
+# description: Prints help information
+#
+# arguments:   none
+#
+# returns: none
+#
+
+def Usage():
+print("Syntax:  %s [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1'] [-d 
'rootDirectory2'] [-d 'rootDirectory3']... [-q e|w] \
+'rootDirectory0' 'uqiFile'|'uqiFileDirectory' ['excludedDirectory1'] 
['excludedDirectory2'] ['excludedDirectory3']...\n%s" %
+  (os.path.basename(sys.argv[0]),
+   """\nFunction will sync up UQI definitions with uni files based on 
vfi/vfr/hfr/sd/sdi in the tree.\n
+ Required Arguments:
+   'rootdirectory0'   path to root directory
+   'uqiFileDirectory' path to UQI file(UqiList.uni)
+   'uqiFile'  UQI file
+
+ Options:
+   -hShow this help
+   -bBuild option returns error if any new UQI needs 
assigning
+ based on vfi/vfr/hfr/sd/sdi when no -u option is 
specified
+   -uCreate new UQIs that does not already exist in 
uqiFile for
+ any string requiring a UQI based on 
vfi/vfr/hfr/sd/sdi
+ NOTE: 'uqiFile' cannot be readonly!
+   -lLanguage deletion option (keeps only English and 
uqi)
+ moves all UQIs to 'uqiFile'
+ NOTE: Uni files cannot be readonly!
+   -xExclude 'rootDirectory'/'excludedDirectory1' &
+ 'rootDirectory'/'excludedDirectory2'... from UQI 
list build
+ NOTE: Cannot be the same as rootDirectory
+   -dAdd multiple root directories to process
+   -qPrint warning(w) or return error(e) if different 
HII questions
+ are referring same string token
+
+ Return error if any duplicated UQI string or value in UQI list or if no 
definition
+ for any string referred by HII question when -b or -u is specified
+
+ NOTE: Options must be specified before parameters
+ """))
+sys.exit()
+
+
+# **
+# description: Get uni file encoding
+#
+# arguments:   Filename - name of uni file
+#
+# returns: utf-8 or utf-16
+#
+def GetUniFileEncoding(Filename):
+#
+# Detect Byte Order Mark at beginning of file.  Default to UTF-8
+#
+Encoding = 'utf-8'
+
+#
+# Read file
+#
+try:
+with open(Filename, mode='rb') as UniFile:
+FileIn = UniFile.read()
+except:
+return Encoding
+
+if (FileIn.startswith(codecs.BOM_UTF16_BE) or 
FileIn.startswith(codecs.BOM_UTF16_LE)):
+Encoding = 'utf-16'
+
+return Encoding
+
+
+# rewrite function os.path.walk
+def Walk(Top, Func, Arg):
+try:
+Names = os.listdir(Top)
+except os.error:
+return
+Func(Arg, Top, Names)
+for Name in Names:
+Name = os.path.join(Top, Name)
+if os.path.isdir(Name):
+Walk(Name, Func, Arg)
+
+
+# **
+# description: Parses commandline arguments and options
+#  Calls function processUni to build dictionary of strings
+#  Calls other functions according to user specified 

[edk2-devel] [edk2-platform patch 1/2] Platform/Intel:Add GenBiosId into edk2-platforms/Platform/Intel/Tools

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855

GenBiosId is a tool to generate the BIOS ID binary file which uses
the data from the configuration file.
This tool can be run in both Py2 and Py3.

This patch is going to add GenBiosId to Platform/Intel/Tools

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Ard Biesheuvel 
Cc: Leif Lindholm 
Cc: Michael D Kinney 
Signed-off-by: Zhiju.Fan 
---
 Platform/Intel/Tools/GenBiosId/BiosId.env   |  27 +
 Platform/Intel/Tools/GenBiosId/GenBiosId.py | 180 
 2 files changed, 207 insertions(+)
 create mode 100644 Platform/Intel/Tools/GenBiosId/BiosId.env
 create mode 100644 Platform/Intel/Tools/GenBiosId/GenBiosId.py

diff --git a/Platform/Intel/Tools/GenBiosId/BiosId.env 
b/Platform/Intel/Tools/GenBiosId/BiosId.env
new file mode 100644
index 00..e1e913da76
--- /dev/null
+++ b/Platform/Intel/Tools/GenBiosId/BiosId.env
@@ -0,0 +1,27 @@
+## @file
+#  This file is used to define the BIOS ID parameters of the build.
+#  This file is processed by GenBiosId.
+#  Here, it is just a template and can be customized by user.
+#
+#  BIOS ID string format:
+#
$(BOARD_ID)$(BOARD_REV).$(BOARD_EXT).$(VERSION_MAJOR).$(BUILD_TYPE)$(VERSION_MINOR).YYMMDDHHMM
+#  All fields must have a fixed length. YYMMDDHHMM is UTC time.
+#Example: "EMLATOR1.000.0001.D01.1906141517"
+#
+#  If DATE is specified for YYMMDD and TIME is specified for HHMM like below,
+#  GenBiosId will use the value of DATE and TIME to fill YYMMDDHHMM,
+#  otherwise GenBiosId will fill YYMMDDHHMM with current UTC time of the build 
machine.
+#DATE  = 190614
+#TIME  = 1517
+#
+#  Copyright (c) 2019, Intel Corporation. All rights reserved.
+#  SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+##
+[config]
+BOARD_ID  = KBLRVP3
+BOARD_REV = 1
+BOARD_EXT = 000
+BUILD_TYPE= D
+VERSION_MAJOR = 0001
+VERSION_MINOR = 01
diff --git a/Platform/Intel/Tools/GenBiosId/GenBiosId.py 
b/Platform/Intel/Tools/GenBiosId/GenBiosId.py
new file mode 100644
index 00..20fb7592b4
--- /dev/null
+++ b/Platform/Intel/Tools/GenBiosId/GenBiosId.py
@@ -0,0 +1,180 @@
+## @file
+# Trim files preprocessed by compiler
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+##
+# Import Modules
+#
+import os
+import sys
+import time
+import logging
+import struct
+import datetime
+import argparse
+import platform
+
+try:
+from configparser import ConfigParser
+except:
+from ConfigParser import ConfigParser
+
+# Config message
+_BIOS_Signature = "$IBIOSI$"
+_SectionKeyName = '__name__'
+_SectionName = 'config'
+
+_ConfigItem = {
+"BOARD_ID": {'Value': '', 'Length': 7},
+"BOARD_REV": {'Value': '', 'Length': 1},
+"BOARD_EXT": {'Value': '', 'Length': 3},
+"BUILD_TYPE": {'Value': '', 'Length': 1},
+"VERSION_MAJOR": {'Value': '', 'Length': 4},
+"VERSION_MINOR": {'Value': '00', 'Length': 2},
+}
+
+# Version message
+__prog__ = 'GenBiosld'
+__description__ = 'Trim files preprocessed by compiler'
+__copyright__ = 'Copyright (c) 2019, Intel Corporation. All rights 
reserved. '
+__version__ = '%s Version %s' % (__prog__, '0.1 ')
+
+# ExtraData message
+_Usage = "Usage: GenBiosId -i Configfile -o OutputFile [-ot OutputTextFile]"
+_ConfigSectionNotDefine = "Not support the config file format, need config 
section"
+_ErrorMessageTemplate = '\n\n%(tool)s...\n : error: %(msg)s\n\t%(extra)s'
+_ErrorLogger = logging.getLogger("tool_error")
+_ErrorFormatter = logging.Formatter("%(message)s")
+_ConfigLenInvalid = "Config item %s length is invalid"
+_ConfigItemInvalid = "Item %s is invalid"
+
+# Error message
+INFO = 20
+ERRORCODE = 50
+OPTION_MISSING = 'Missing option'
+FORMAT_INVALID = 'Invalid syntax/format'
+FILE_NOT_FOUND = 'File/directory not found in workspace'
+FORMAT_UNKNOWN_ERROR = 'Unknown error in syntax/format'
+FORMAT_NOT_SUPPORTED = 'Not supported syntax/format'
+
+
+def SetEdkLogger():
+_ErrorLogger.setLevel(INFO)
+_ErrorCh = logging.StreamHandler(sys.stderr)
+_ErrorCh.setFormatter(_ErrorFormatter)
+_ErrorLogger.addHandler(_ErrorCh)
+return _ErrorLogger
+
+
+# Output the error message and exit the tool
+def EdkLogger(ToolName, Message, ExtraData):
+_ErrorLogger = SetEdkLogger()
+TemplateDict = {"tool": ToolName, "msg": Message, "extra": ExtraData}
+LogText = _ErrorMessageTemplate % TemplateDict
+_ErrorLogger.log(ERRORCODE, LogText)
+sys.exit()
+
+
+# Open the file in the correct way
+def FileOpen(FileName, Mode, Buffer=-1):
+def LongFilePath(FileName):
+FileName = os.path.normpath(FileName)
+if platform.system() == 'Windows':
+if FileName.startswith('?\\'):
+return FileName
+if FileName.startswith(''):
+return '?\\UNC\\' + FileName[2:]
+if os.path.isabs(FileName):
+return '?\\' + FileName
+return FileName
+
+

[edk2-devel] [PATCH] BaseTools:Add import in FvImageSection

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1872

Since MultipleWorkspace is not imported in FvImageSection,
an error of "global name 'MWS' is not defined" appeared.

This patch is going to fix that issue.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/GenFds/FvImageSection.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/BaseTools/Source/Python/GenFds/FvImageSection.py 
b/BaseTools/Source/Python/GenFds/FvImageSection.py
index 2d38ff6096..b2953d822b 100644
--- a/BaseTools/Source/Python/GenFds/FvImageSection.py
+++ b/BaseTools/Source/Python/GenFds/FvImageSection.py
@@ -17,6 +17,7 @@ import subprocess
 from .GenFdsGlobalVariable import GenFdsGlobalVariable
 import Common.LongFilePathOs as os
 from CommonDataClass.FdfClass import FvImageSectionClassObject
+from Common.MultipleWorkspace import MultipleWorkspace as mws
 from Common import EdkLogger
 from Common.BuildToolError import *
 from Common.DataType import *
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#42630): https://edk2.groups.io/g/devel/message/42630
Mute This Topic: https://groups.io/mt/32141194/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH V3] BaseTools:Add DetectNotUsedItem.py to Edk2\BaseTools\Scripts

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1850

This script is used to Detect unreferenced PCD and GUID/Protocols/PPIs.
The input parameters are Dec file and package directory.

This script can be run in both Py2 and Py3.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Scripts/DetectNotUsedItem.py | 185 +
 1 file changed, 185 insertions(+)
 create mode 100644 BaseTools/Scripts/DetectNotUsedItem.py

diff --git a/BaseTools/Scripts/DetectNotUsedItem.py 
b/BaseTools/Scripts/DetectNotUsedItem.py
new file mode 100644
index 00..655fb65a96
--- /dev/null
+++ b/BaseTools/Scripts/DetectNotUsedItem.py
@@ -0,0 +1,185 @@
+## @file
+# Detect unreferenced PCD and GUID/Protocols/PPIs.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+'''
+DetectNotUsedItem
+'''
+import re
+import os
+import sys
+import argparse
+
+#
+# Globals for help information
+#
+__prog__= 'DetectNotUsedItem'
+__version__ = '%s Version %s' % (__prog__, '0.1')
+__copyright__   = 'Copyright (c) 2019, Intel Corporation. All rights reserved.'
+__description__ = "Detect unreferenced PCD and GUID/Protocols/PPIs.\n"
+
+SectionList = ["LibraryClasses", "Guids", "Ppis", "Protocols", "Pcd"]
+
+class PROCESS(object):
+
+  def __init__(self, DecPath, InfDirs):
+self.Dec = DecPath
+self.InfPath = InfDirs
+self.Log = []
+
+  def ParserDscFdfInfFile(self):
+AllContentList = []
+for File in (self.SearchbyExt(".dsc") + self.SearchbyExt(".fdf") + 
self.SearchbyExt(".inf")):
+  AllContentList += self.ParseDscFdfInfContent(File)
+return AllContentList
+
+  #Search File by extension name
+  def SearchbyExt(self, Ext):
+FileList = []
+for path in self.InfPath:
+  for root, _, files in os.walk(path, topdown=True, followlinks=False):
+for filename in files:
+  if filename.endswith(Ext):
+FileList.append(os.path.join(root, filename))
+return FileList
+
+  # Parse DEC file to get Line number and Name
+  # return section name, the Item Name and comments line number
+  def ParseDecContent(self):
+SectionRE = re.compile(r'\[(.*)\]')
+Flag = False
+Comments ={}
+Comment_Line = []
+ItemName = {}
+with open(self.Dec, 'r') as F:
+  for Index, content in enumerate(F):
+NotComment = not content.strip().startswith("#")
+Section = SectionRE.findall(content)
+if Section and NotComment:
+  Flag = self.IsNeedParseSection(Section[0])
+if Flag:
+  Comment_Line.append(Index)
+  if NotComment:
+if content != "\n" and content != "\r\n":
+  ItemName[Index] = 
content.split('=')[0].split('|')[0].split('#')[0].strip()
+  Comments[Index] = Comment_Line
+  Comment_Line = []
+return ItemName, Comments
+
+  def IsNeedParseSection(self, SectionName):
+for item in SectionList:
+  if item in SectionName:
+return True
+return False
+
+  #Parse DSC, FDF, INF File, remove comments, return Lines list
+  def ParseDscFdfInfContent(self, File):
+with open(File,'r') as F:
+  lines = F.readlines()
+for Index in range(len(lines)-1, -1, -1):
+  if lines[Index].strip().startswith("#") or lines[Index] == "\n" or 
lines[Index] == "\r\n":
+lines.remove(lines[Index])
+  elif "#" in lines[Index]:
+lines[Index] = lines[Index].split("#")[0].strip()
+  else:
+lines[Index] = lines[Index].strip()
+return lines
+
+  def DetectNotUsedItem(self):
+NotUsedItem = {}
+DecItem, DecComments = self.ParseDecContent()
+InfDscFdfContent = self.ParserDscFdfInfFile()
+for LineNum in list(DecItem.keys()):
+  DecItemName = DecItem[LineNum]
+  Match_reg = re.compile("(?

[edk2-devel] [PATCH V3] BaseTools:add UniTool.py to Edk2\BaseTools\Scripts

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855

UniTool is one python script to generate UQI (Universal Question
Identifier) unicode string for HII question PROMPT string. UQI
string can be used to identify each HII question.
The scripts function will sync up UQI definitions with uni files
based on vfi/vfr/hfr/sd/sdi in the tree.

This script can be run in both Py2 and Py3.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Scripts/UniTool.py | 514 +++
 1 file changed, 514 insertions(+)
 create mode 100644 BaseTools/Scripts/UniTool.py

diff --git a/BaseTools/Scripts/UniTool.py b/BaseTools/Scripts/UniTool.py
new file mode 100644
index 00..1064e261f7
--- /dev/null
+++ b/BaseTools/Scripts/UniTool.py
@@ -0,0 +1,514 @@
+## @file
+# generate UQI (Universal Question Identifier) unicode string for HII question 
PROMPT string. UQI string can be used to
+# identify each HII question.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+import re
+import sys
+import os
+import getopt
+import codecs
+import fnmatch
+import logging
+
+# global variable declarations
+QuestionError = False
+UqiList = re.compile('^#string[ \t]+([A-Z_0-9]+)[ \t]+#language[ \t]+uqi[ 
\t\r\n]+"(?:[x\S]{1,2})([0-9a-fA-F]{4,5})"',
+ re.M).findall
+AllUqis = {}
+StringDict = {}
+GlobalVarId = {}
+Options = {}
+
+
+# **
+# description: Prints help information
+#
+# arguments:   none
+#
+# returns: none
+#
+
+def Usage():
+print("Syntax:  %s [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1'] [-d 
'rootDirectory2'] [-d 'rootDirectory3']... [-q e|w] \
+'rootDirectory0' 'uqiFile'|'uqiFileDirectory' ['excludedDirectory1'] 
['excludedDirectory2'] ['excludedDirectory3']...\n%s" %
+ (os.path.basename(sys.argv[0]),
+  """\nFunction will sync up UQI definitions with uni files based 
on vfi/vfr/hfr/sd/sdi in the tree.\n
+Required Arguments:
+  'rootdirectory0'   path to root directory
+  'uqiFileDirectory' path to UQI file(UqiList.uni)
+  'uqiFile'  UQI file
+
+Options:
+  -hShow this help
+  -bBuild option returns error if any new UQI 
needs assigning
+based on vfi/vfr/hfr/sd/sdi when no -u option 
is specified
+  -uCreate new UQIs that does not already exist in 
uqiFile for
+any string requiring a UQI based on 
vfi/vfr/hfr/sd/sdi
+NOTE: 'uqiFile' cannot be readonly!
+  -lLanguage deletion option (keeps only English 
and uqi)
+moves all UQIs to 'uqiFile'
+NOTE: Uni files cannot be readonly!
+  -xExclude 'rootDirectory'/'excludedDirectory1' &
+'rootDirectory'/'excludedDirectory2'... from 
UQI list build
+NOTE: Cannot be the same as rootDirectory
+  -dAdd multiple root directories to process
+  -qPrint warning(w) or return error(e) if 
different HII questions
+are referring same string token
+
+Return error if any duplicated UQI string or value in UQI list or if 
no definition
+for any string referred by HII question when -b or -u is specified
+
+NOTE: Options must be specified before parameters
+"""))
+sys.exit()
+
+
+# **
+# description: Get uni file encoding
+#
+# arguments:   Filename - name of uni file
+#
+# returns: utf-8 or utf-16
+#
+def GetUniFileEncoding(Filename):
+#
+# Detect Byte Order Mark at beginning of file.  Default to UTF-8
+#
+Encoding = 'utf-8'
+
+#
+# Read file
+#
+try:
+with open(Filename, mode='rb') as UniFile:
+FileIn = UniFile.read()
+except:
+return Encoding
+
+if (FileIn.startswith(codecs.BOM_UTF16_BE) or 
FileIn.startswith(codecs.BOM_UTF16_LE)):
+Encoding = 'utf-16'
+
+return Encoding
+
+
+# rewrite function os.path.walk
+def Walk(Top, Func, Arg):
+try:
+Names = os.listdir(Top)
+except os.error:
+return
+Func(Arg, Top, Names)
+for Name in Names:
+Name = os.path.join(Top, Name)
+if os.path.isdir(Name):
+Walk(Name, Func, Arg)
+
+
+# **
+# description: Parses commandline arguments and options
+#  Calls function processUni to build dictionary of strings
+#  Calls other functions according to user specified options
+#
+# arguments:   argv 

[edk2-devel] [PATCH] BaseTools:Remove unused Edk2\BuildNotes2.txt

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1889

Edk2\BuildNotes2.txt is not maintained any more. User can find the
related info in wiki. So, this file can be removed.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BuildNotes2.txt | 154 
 1 file changed, 154 deletions(-)
 delete mode 100644 BuildNotes2.txt

diff --git a/BuildNotes2.txt b/BuildNotes2.txt
deleted file mode 100644
index 8bf4f6930a..00
--- a/BuildNotes2.txt
+++ /dev/null
@@ -1,154 +0,0 @@
-Intel(R) Platform Innovation Framework for EFI
-EFI Development Kit II (EDK II)
-2011-12-14
-
-Intel is a trademark or registered trademark of Intel Corporation or its
-subsidiaries in the United States and other countries.
-* Other names and brands may be claimed as the property of others.
-Copyright (c) 2007 - 2011, Intel Corporation. All rights reserved.
-
-EDK II packages can be gathered from the following address:
-  https://github.com/tianocore/edk2.git
-
-The detailed introduction of these packages can be found in each package
-description file. (The *.dec file under the package directory)
-
-

-The most recent version of the setup instructions is available on the EDK II
-web-site:
-  
https://github.com/tianocore/tianocore.github.io/wiki/Getting%20Started%20with%20EDK%20II
-
-

-Quick Start (Windows Development Platform)

-
-In a command prompt window, change to the top-level directory of the EDK II 
-source.
-
-Note:
-  The first time the edksetup script is executed, it creates three files in the
-  %WORKSPACE%\Conf directory. The files: tools_def.txt, target.txt and
-  build_rule.txt, are only created if they do not exist, if they exist, they
-  are not touched.
-
-First, set up your project workspace. If you have previously initialized this
-WORKSPACE, and a newer version of the *.template files in
-WORKSPACE\BaseTools\Conf exists, remove the *.txt files in the WORKSPACE\Conf
-directory prior to running the edksetup script.
-
-For the reference build of the Nt32 Platform emulation environment, use the 
-edksetup.bat option: --nt32. For building other platforms or modules, this
-option is not required, as Visual Studio standard includes, libraries and/or
-dlls are not required for normal development.
-
-c:\MyWork\edk2\> edksetup --nt32
-
-The default tool chain (named MYTOOLS) is pre-configured to use VS2008 for IA32
-and X64 target architectures and DDK3790 for IPF target architectures. To use a
-different tool chain, either modify the tools_def.txt file's MYTOOLS entries,
-or modify the %WORKSPACE%\Conf\target.txt file's TOOL_CHAIN_TAG. The 
pre-defined
-tags are listed near the top of the %WORKSPACE%\Conf\tools_def.txt file, below
-the Supported Tool Chains comment.
-Alternatively, you may use the build command's -t option to specify a different
-tool chain tag name: build -t VS2008 ... , for example.  Using this method will
-require that you always use the build command's -t option. If you use 64-bit 
-Windows OS, you should use tool chain tag name with x86, such as VS2008x86.
-
-
-Next, go to the module directory and begin to build. This example is for the
-HelloWorld application.
-
-c:\MyWork\edk2\> cd MdeModulePkg\Application\HelloWorld
-c:\MyWork\edk2\> build
-
-If you want to build the a module in another package (for example, 
-MdePkg\Library\BaseLib\BaseLib.inf), please edit the file
-%WORKSPACE%\Conf\Target.txt first.
-
-Change the following line
-ACTIVE_PLATFORM   = MdeModulePkg/MdeModulePkg.dsc
-to
-ACTIVE_PLATFORM   = MdePkg/MdePkg.dsc
-
-Then go to MdePkg\Library\BaseLib directory and type build:
-c:\MyWork\edk2\> cd MdePkg\Library\BaseLib
-c:\MyWork\edk2\> build
-
-If you want build a platform, ACTIVE_PLATFORM must be set to your desired
-platform dsc file, go to directory which must be not a module's directory, and
-run "build" command.
-
-Instead of changing Target.txt, you can specify platform, module and/or
-architecture on command line.
-For example, if you want to build NT32 platform, you can just type 
-
-c:\MyWork\edk2\> build -p Nt32Pkg\Nt32Pkg.dsc -a IA32
-
-and if you want to build HelloWorld module, you can just type
-
-c:\MyWork\edk2\> build -p Nt32Pkg\Nt32Pkg.dsc -a IA32 -m 
MdeModulePkg\Application\HelloWorld\HelloWorld.inf
-
-Other helpful command line options of build tool include "-v" and "-d".
-The "-v" option is used to turn on the verbose build, which provide more
-information during the build. "-d " option is used to
-turn on the debug information which is helpful debugging build tools.
-
-For more information on build options, please try "build -h" on command line.
-
-Note:
-  The Windows style help option "/?" is not a valid option for the build
-  command.
-
-

[edk2-devel] [PATCH V2] BaseTools/GenBiosId: Add a new tool GenBiosId

[Fan, ZhijuX]

GenBiosId is a tool to generate the BIOS ID binary file which uses
the data from the configuration file.
https://bugzilla.tianocore.org/show_bug.cgi?id=1846

v2:v1 is a tool of C type and v2 is python type.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Shenglei Zhang 
---
 BaseTools/BinWrappers/PosixLike/GenBiosId   |  14 +++
 BaseTools/BinWrappers/WindowsLike/GenBiosId.bat |   3 +
 BaseTools/Source/Python/GenBiosId/BiosId.env|  27 ++
 BaseTools/Source/Python/GenBiosId/GenBiosId.py  | 118 
 4 files changed, 162 insertions(+)
 create mode 100644 BaseTools/BinWrappers/PosixLike/GenBiosId
 create mode 100644 BaseTools/BinWrappers/WindowsLike/GenBiosId.bat
 create mode 100644 BaseTools/Source/Python/GenBiosId/BiosId.env
 create mode 100644 BaseTools/Source/Python/GenBiosId/GenBiosId.py

diff --git a/BaseTools/BinWrappers/PosixLike/GenBiosId 
b/BaseTools/BinWrappers/PosixLike/GenBiosId
new file mode 100644
index 00..1dd28e9662
--- /dev/null
+++ b/BaseTools/BinWrappers/PosixLike/GenBiosId
@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+#python `dirname $0`/RunToolFromSource.py `basename $0` $*
+
+# If a ${PYTHON_COMMAND} command is available, use it in preference to python
+if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then
+python_exe=${PYTHON_COMMAND}
+fi
+
+full_cmd=${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028 for a 
discussion of why $0 is not a good choice here
+dir=$(dirname "$full_cmd")
+exe=$(basename "$full_cmd")
+
+export PYTHONPATH="$dir/../../Source/Python${PYTHONPATH:+:"$PYTHONPATH"}"
+exec "${python_exe:-python}" "$dir/../../Source/Python/$exe/$exe.py" "$@"
diff --git a/BaseTools/BinWrappers/WindowsLike/GenBiosId.bat 
b/BaseTools/BinWrappers/WindowsLike/GenBiosId.bat
new file mode 100644
index 00..e1f61382c8
--- /dev/null
+++ b/BaseTools/BinWrappers/WindowsLike/GenBiosId.bat
@@ -0,0 +1,3 @@
+@setlocal
+@set ToolName=%~n0%
+@%PYTHON_COMMAND% %BASE_TOOLS_PATH%\Source\Python\%ToolName%\%ToolName%.py %*
diff --git a/BaseTools/Source/Python/GenBiosId/BiosId.env 
b/BaseTools/Source/Python/GenBiosId/BiosId.env
new file mode 100644
index 00..e1e913da76
--- /dev/null
+++ b/BaseTools/Source/Python/GenBiosId/BiosId.env
@@ -0,0 +1,27 @@
+## @file
+#  This file is used to define the BIOS ID parameters of the build.
+#  This file is processed by GenBiosId.
+#  Here, it is just a template and can be customized by user.
+#
+#  BIOS ID string format:
+#
$(BOARD_ID)$(BOARD_REV).$(BOARD_EXT).$(VERSION_MAJOR).$(BUILD_TYPE)$(VERSION_MINOR).YYMMDDHHMM
+#  All fields must have a fixed length. YYMMDDHHMM is UTC time.
+#Example: "EMLATOR1.000.0001.D01.1906141517"
+#
+#  If DATE is specified for YYMMDD and TIME is specified for HHMM like below,
+#  GenBiosId will use the value of DATE and TIME to fill YYMMDDHHMM,
+#  otherwise GenBiosId will fill YYMMDDHHMM with current UTC time of the build 
machine.
+#DATE  = 190614
+#TIME  = 1517
+#
+#  Copyright (c) 2019, Intel Corporation. All rights reserved.
+#  SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+##
+[config]
+BOARD_ID  = EMLATOR
+BOARD_REV = 1
+BOARD_EXT = 000
+BUILD_TYPE= D
+VERSION_MAJOR = 0001
+VERSION_MINOR = 01
diff --git a/BaseTools/Source/Python/GenBiosId/GenBiosId.py 
b/BaseTools/Source/Python/GenBiosId/GenBiosId.py
new file mode 100644
index 00..8259b17afd
--- /dev/null
+++ b/BaseTools/Source/Python/GenBiosId/GenBiosId.py
@@ -0,0 +1,118 @@
+## @file
+# Trim files preprocessed by compiler
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+##
+# Import Modules
+#
+import Common.LongFilePathOs as os
+import sys
+import struct
+import time
+import datetime
+import argparse
+try:
+from configparser import ConfigParser
+except:
+from ConfigParser import ConfigParser
+from Common.BuildToolError import *
+from Common.Misc import *
+from Common.DataType import *
+from Common.BuildVersion import gBUILD_VERSION
+import Common.EdkLogger as EdkLogger
+from Common.LongFilePathSupport import OpenLongFilePath as open
+
+_BIOS_Signature = "$IBIOSI$"
+_SectionKeyName = '__name__'
+_SectionName = 'config'
+
+__prog__ = 'GenBiosld'
+__description__ = 'Trim files preprocessed by compiler'
+__copyright__ = 'Copyright (c) 2019, Intel Corporation. All rights 
reserved. '
+__version__ = '%s Version %s' % (__prog__, '0.1 ')
+
+_ConfigItem = {
+"BOARD_ID"   : {'Value' : '', 'Length' : 7},
+"BOARD_REV"  : {'Value' : '', 'Length' : 1},
+"BOARD_EXT"  : { 'Value' : '', 'Length' : 3},
+"BUILD_TYPE" : {'Value' : '', 'Length' :1},
+"VERSION_MAJOR" : {'Value' : '', 'Length' : 4},
+"VERSION_MINOR" : {'Value' : '00', 'Length' : 2},
+
+}
+
+
+_Usage = "Usage: GenBiosId -i Configfile -o OutputFile [-ob OutputBatchFile]"
+_ConfigSectionNotDefine = "Not support the config file format, need config 
section"
+_ConfigLenInvalid = "Config item %s length is invalid"

[edk2-devel] [PATCH 2/3 V2] Platform/Intel:Add build parameter to support Binary Cache

[Fan, ZhijuX]

Need extend the options in the Intel/build_bios.py file
to support Binary Cache.

 --hash:
   Enable hash-based caching during build process.
 --binary-destination:
   Generate a cache of binary files in the specified directory.
 --binary-source:
   Consume a cache of binary files from the specified directory.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Steven Shi 
Cc: Shifei A Lu 
Cc: Xiaohu Zhou 
Cc: Isaac W Oram 
Cc: Chasel Chiu 
Cc: Michael Kubacki 
Cc: Nate DeSimone 
Signed-off-by: Zhiju.Fan 
---
 Platform/Intel/build_bios.py | 27 +++
 1 file changed, 27 insertions(+)

diff --git a/Platform/Intel/build_bios.py b/Platform/Intel/build_bios.py
index 09eceddeff..c01b953d16 100644
--- a/Platform/Intel/build_bios.py
+++ b/Platform/Intel/build_bios.py
@@ -343,6 +343,7 @@ def build(config):
 print(" SILENT_MODE= ", config.get("SILENT_MODE"))
 print(" REBUILD_MODE   = ", config.get("REBUILD_MODE"))
 print(" BUILD_ROM_ONLY = ", config.get("BUILD_ROM_ONLY"))
+print(" BINARY_CACHE_CMD_LINE = ", config.get("HASH"), 
config.get("BINARY_CACHE_CMD_LINE"))
 print("==")
 
 command = ["build", "-n", config["NUMBER_OF_PROCESSORS"]]
@@ -353,6 +354,10 @@ def build(config):
 if config["EXT_BUILD_FLAGS"] and config["EXT_BUILD_FLAGS"] != "":
 command.append(config["EXT_BUILD_FLAGS"])
 
+if config.get('BINARY_CACHE_CMD_LINE'):
+command.append(config['HASH'])
+command.append(config['BINARY_CACHE_CMD_LINE'])
+
 if config.get("SILENT_MODE", "FALSE") == "TRUE":
 command.append("--silent")
 command.append("--quiet")
@@ -858,6 +863,17 @@ def get_cmd_config_arguments(arguments):
 if arguments.fspapi is True:
 result["API_MODE_FSP_WRAPPER_BUILD"] = "TRUE"
 
+if not arguments.UseHashCache:
+result['BINARY_CACHE_CMD_LINE'] = ''
+elif arguments.BinCacheDest:
+result['HASH'] = '--hash'
+result['BINARY_CACHE_CMD_LINE'] = '--binary-destination=%s' % 
arguments.BinCacheDest
+elif arguments.BinCacheSource:
+result['HASH'] = '--hash'
+result['BINARY_CACHE_CMD_LINE'] = '--binary-source=%s' % 
arguments.BinCacheSource
+else:
+result['BINARY_CACHE_CMD_LINE'] = ''
+
 return result
 
 
@@ -934,6 +950,17 @@ def get_cmd_arguments(build_config):
 parser.add_argument("--fspapi", help="API mode fsp wrapper build enabled",
 action='store_true', dest="fspapi")
 
+parser.add_argument("--hash", action="store_true", dest="UseHashCache", 
default=False,
+help="Enable hash-based caching during build process.")
+
+parser.add_argument("--binary-destination", help="Generate a cache of 
binary \
+files in the specified directory.",
+action='store', dest="BinCacheDest")
+
+parser.add_argument("--binary-source", help="Consume a cache of binary 
files \
+from the specified directory.",
+action='store', dest="BinCacheSource")
+
 return parser.parse_args()
 
 
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#42482): https://edk2.groups.io/g/devel/message/42482
Mute This Topic: https://groups.io/mt/32091878/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH 1/3 V2] Intel/Readme.md:Add instructions about Binary Cache in Readme.md

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1784
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1785

Add detailed instructions about Binary Cache in Readme.md,
Extend options to support Binary Cache in the Kabylake
build bld.bat file, Purley build bld.bat file, build_bios.py

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Steven Shi 
Cc: Shifei A Lu 
Cc: Xiaohu Zhou 
Cc: Isaac W Oram 
Cc: Chasel Chiu 
Cc: Michael Kubacki 
Cc: Nate DeSimone 
Signed-off-by: Zhiju.Fan 
---
 Platform/Intel/Readme.md | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/Platform/Intel/Readme.md b/Platform/Intel/Readme.md
index cc0151066e..d95d0fa54e 100644
--- a/Platform/Intel/Readme.md
+++ b/Platform/Intel/Readme.md
@@ -134,6 +134,9 @@ return back to the minimum platform caller.
   | --performance | performance build enabled   |
   | --fsp | fsp wrapper build enabled   |
   | --fspapi  | API mode fsp wrapper build enabled  |
+  | --hash| Enable hash-based caching   |
+  | --binary-destination  | create cache in specified directory |
+  | --binary-source   | Consume cache from directory|
   | |
 
 * For more information on build options
@@ -196,7 +199,9 @@ For PurleyOpenBoardPkg
 2. Type "cd edk2-platforms\Platform\Intel\PurleyOpenBoardPkg\BoardMtOlympus".
 3. Type "GitEdk2MinMtOlympus.bat" to setup GIT environment.
 4. Type "bld" to build Purley Mt Olympus board UEFI firmware image, "bld 
release" for release build, "bld clean" to
-   remove intermediate files.
+   remove intermediate files."bld cache-produce" Generate a cache of binary 
files in the specified directory,
+   "bld cache-consume" Consume a cache of binary files from the specified 
directory, BINARY_CACHE_PATH is empty,
+   used "BinCache" as default path.
 
 The validated version of iasl compiler that can build MinPurley is 20180629. 
Older version may generate ACPI build
 errors.
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#42481): https://edk2.groups.io/g/devel/message/42481
Mute This Topic: https://groups.io/mt/32091876/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH 0/3] Makes binary cache available in Platform/Intel

[Fan, ZhijuX]

Add detailed instructions about Binary Cache in Readme.md,
Extend options to support Binary Cache in the Kabylake
build bld.bat file, Purley build bld.bat file, build_bios.py

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Steven Shi 
Cc: Shifei A Lu 
Cc: Xiaohu Zhou 
Cc: Isaac W Oram 
Cc: Chasel Chiu 
Cc: Michael Kubacki 
Cc: Nate DeSimone 
Signed-off-by: Zhiju.Fan 

*** BLURB HERE ***

Fan, Zhiju (3):
  Intel/Readme.md:Add instructions about Binary Cache in Readme.md
  Platform/Intel:Add build parameter to support Binary Cache
  PurleyOpenBoardPkg:Extend options in bld.bat to support Binary Cache

 Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/bld.bat  | 17 
+++--
 Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/prebuild.bat |  4 ++--
 Platform/Intel/Readme.md  |  7 ++-
 Platform/Intel/build_bios.py  | 27 
+++
 4 files changed, 50 insertions(+), 5 deletions(-)

-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#42480): https://edk2.groups.io/g/devel/message/42480
Mute This Topic: https://groups.io/mt/32091867/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH V2] BaseTools:Add DetectNotUsedItem.py to Edk2\BaseTools\Scripts

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1850

This script is used to Detect unreferenced PCD and GUID/Protocols/PPIs.
The input parameters are Dec file and package directory.

This script can be run in both Py2 and Py3.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Scripts/DetectNotUsedItem.py | 199 +
 1 file changed, 199 insertions(+)
 create mode 100644 BaseTools/Scripts/DetectNotUsedItem.py

diff --git a/BaseTools/Scripts/DetectNotUsedItem.py 
b/BaseTools/Scripts/DetectNotUsedItem.py
new file mode 100644
index 00..7e3568fcf9
--- /dev/null
+++ b/BaseTools/Scripts/DetectNotUsedItem.py
@@ -0,0 +1,199 @@
+## @file
+# Detect unreferenced PCD and GUID/Protocols/PPIs.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+'''
+DetectNotUsedItem
+'''
+import re
+import os
+import sys
+import argparse
+
+#
+# Globals for help information
+#
+__prog__ = 'DetectNotUsedItem'
+__version__ = '%s Version %s' % (__prog__, '0.1')
+__copyright__ = 'Copyright (c) 2019, Intel Corporation. All rights reserved.'
+__description__ = "Detect unreferenced PCD and GUID/Protocols/PPIs.\n"
+
+SectionList = ["LibraryClasses", "Guids", "Ppis", "Protocols", "Pcd"]
+
+
+class PROCESS(object):
+
+def __init__(self, DecPath, InfDirs):
+self.Dec = DecPath
+self.InfPath = InfDirs
+self.Log = []
+
+def ParserDscFdfInfFile(self):
+AllContentList = []
+for File in self.SearchbyExt([".dsc", ".fdf", ".inf"]):
+AllContentList += self.ParseDscFdfInfContent(File)
+return AllContentList
+
+# Search File by extension name
+def SearchbyExt(self, ExtList):
+FileList = []
+for path in self.InfPath:
+if type(ExtList) == type(''):
+for root, _, files in os.walk(path, topdown=True, 
followlinks=False):
+for filename in files:
+if filename.endswith(ExtList):
+FileList.append(os.path.join(root, filename))
+elif type(ExtList) == type([]):
+for root, _, files in os.walk(path, topdown=True, 
followlinks=False):
+for filename in files:
+for Ext in ExtList:
+if filename.endswith(Ext):
+FileList.append(os.path.join(root, filename))
+return FileList
+
+# Parse DEC file to get Line number and Name
+# return section name, the Item Name and comments line number
+def ParseDecContent(self):
+SectionRE = re.compile(r'\[(.*)\]')
+Flag = False
+Comments = {}
+Comment_Line = []
+ItemName = {}
+with open(self.Dec, 'r') as F:
+for Index, content in enumerate(F):
+NotComment = not content.strip().startswith("#")
+Section = SectionRE.findall(content)
+if Section and NotComment:
+Flag = self.IsNeedParseSection(Section[0])
+if Flag:
+Comment_Line.append(Index)
+if NotComment:
+if content != "\n" and content != "\r\n":
+ItemName[Index] = 
content.split('=')[0].split('|')[0].split('#')[0].strip()
+Comments[Index] = Comment_Line
+Comment_Line = []
+return ItemName, Comments
+
+def IsNeedParseSection(self, SectionName):
+for item in SectionList:
+if item in SectionName:
+return True
+return False
+
+# Parse DSC, FDF, INF File, remove comments, return Lines list
+def ParseDscFdfInfContent(self, File):
+with open(File, 'r') as F:
+lines = F.readlines()
+for Index in range(len(lines) - 1, -1, -1):
+if lines[Index].strip().startswith("#") or lines[Index] == "\n" or 
lines[Index] == "\r\n":
+lines.remove(lines[Index])
+elif "#" in lines[Index]:
+lines[Index] = lines[Index].split("#")[0].strip()
+else:
+lines[Index] = lines[Index].strip()
+return lines
+
+def DetectNotUsedItem(self):
+NotUsedItem = {}
+DecItem, DecComments = self.ParseDecContent()
+InfDscFdfContent = self.ParserDscFdfInfFile()
+for LineNum in list(DecItem.keys()):
+DecItemName = DecItem[LineNum]
+Match_reg = re.compile("(?

[edk2-devel] [PATCH] BaseTools:Build cache support the cache files for library package

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1867

Current build cache cannot store the cache for library package.
build -p MdePkg\MdePkg.dsc -a IA32 -b DEBUG -t VS2015x86 --hash
--binary-destination=BinCache
After build, the expected result is the BinCache folder is generated
and the MdePkg build cache files (e.g. .hash and .lib) are stored in
the BinCache folder. But the BinCache folder is not generated at all.

This patch is going to fix that issue.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Steven Shi 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 4 
 1 files changed, 4 insertions(+)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py 
b/BaseTools/Source/Python/AutoGen/AutoGen.py
index a879b6259f..b8ecf3826f 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -3571,6 +3571,10 @@ class ModuleAutoGen(AutoGen):
 
 # Skip the following code for libraries
 if self.IsLibrary:
+try:
+self.CopyModuleToCache()
+except:
+pass
 return
 
 # Skip the following code for modules with no source files
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#41970): https://edk2.groups.io/g/devel/message/41970
Mute This Topic: https://groups.io/mt/31945498/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH] BaseTools:Build Cache output notification message

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1868

Build need output the cache miss or hit notification
message when consume the build cache. Current build does not
output any message which is not clear for user to know
whether the module built result is from cache or not.

This patch adds message about the cache miss or hit when
build is to consume the build cache.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Steven Shi 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py 
b/BaseTools/Source/Python/AutoGen/AutoGen.py
index f1b8f9364b..a3de730d1f 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -3957,7 +3957,9 @@ class ModuleAutoGen(AutoGen):
 shutil.copy(File, destination_dir)
 if self.Name == "PcdPeim" or self.Name == "PcdDxe":
 CreatePcdDatabaseCode(self, TemplateString(), 
TemplateString())
+EdkLogger.quiet("Cache hit ... %s[%s]" % 
(self.MetaFile.Path,self.Arch))
 return True
+EdkLogger.quiet("Cache miss ... %s[%s]" % (self.MetaFile.Path, 
self.Arch))
 return False
 
 ## Create makefile for the module and its dependent libraries
-- 
2.17.1.windows.2


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#41968): https://edk2.groups.io/g/devel/message/41968
Mute This Topic: https://groups.io/mt/31945437/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH 4/4] PurleyOpenBoardPkg:Extend options in bld.bat to support Binary Cache

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1785

Need extend the options in the Kabylake build bld.bat file to
support Binary Cache.

BINARY_CACHE_PATH:
if BINARY_CACHE_PATH is empty, use BinCache as default path

Add "cache-produce" and "cache-consume" to command line,
Used to generate and use Binary Cache files.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Steven Shi 
Cc: Shifei A Lu 
Cc: Xiaohu Zhou 
Cc: Isaac W Oram 
---
 Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/bld.bat  | 21 
+++--
 Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/prebuild.bat |  4 ++--
 2 files changed, 21 insertions(+), 4 deletions(-)

diff --git a/Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/bld.bat 
b/Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/bld.bat
index f624be03a9..f02472239d 100644
--- a/Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/bld.bat
+++ b/Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/bld.bat
@@ -10,6 +10,11 @@ REM Run setlocal to take a snapshot of the environment 
variables.  endlocal is c
 setlocal
 set SCRIPT_ERROR=0
 
+@if not defined BINARY_CACHE_PATH (
+  echo Info: BINARY_CACHE_PATH is empty, use BinCache as default
+  SET BINARY_CACHE_PATH=BinCache
+)
+
 REM  Do NOT use :: for comments Inside of code blocks() 
 
 ::**
@@ -28,6 +33,18 @@ if /I "%1"=="clean" (
   goto :EOF
 )
 
+@if "%~1" == "cache-produce" (
+  SET BINARY_CACHE_CMD_LINE= --hash --binary-destination=%BINARY_CACHE_PATH%
+  shift
+  goto BUILD_FLAGS_LOOP
+)
+
+@if "%~1" == "cache-consume" (
+  SET BINARY_CACHE_CMD_LINE= --hash --binary-source=%BINARY_CACHE_PATH%
+  shift
+  goto BUILD_FLAGS_LOOP
+)
+
 shift
 GOTO :parseCmdLine
 
@@ -86,8 +103,8 @@ echo  Build Start
 echo.
 echo 
 echo.
-echo build %BUILD_CMD_LINE% --log=%BUILD_LOG% %BUILD_REPORT_FLAGS%
-call build %BUILD_CMD_LINE% --log=%BUILD_LOG% %BUILD_REPORT_FLAGS%
+echo build %BUILD_CMD_LINE% --log=%BUILD_LOG% %BUILD_REPORT_FLAGS% 
%BINARY_CACHE_CMD_LINE%
+call build %BUILD_CMD_LINE% --log=%BUILD_LOG% %BUILD_REPORT_FLAGS% 
%BINARY_CACHE_CMD_LINE%
 echo 
 echo.
 echo  Build End
diff --git a/Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/prebuild.bat 
b/Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/prebuild.bat
index d9c1442ea1..4aba30adce 100644
--- a/Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/prebuild.bat
+++ b/Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/prebuild.bat
@@ -182,8 +182,8 @@ set PRE_BUILD_CMD_LINE=%BUILD_CMD_LINE% -D 
MAX_SOCKET=%MAX_SOCKET%
 set PRE_BUILD_LOG=%WORKSPACE%\Build\prebuild.log
 set PRE_BUILD_REPORT=%WORKSPACE%\Build\preBuildReport.txt
 
-echo build %PRE_BUILD_CMD_LINE% -m %BOARD_PKG%\Acpi\BoardAcpiDxe\Dsdt.inf -y 
%PRE_BUILD_REPORT% --log=%PRE_BUILD_LOG%
-call build %PRE_BUILD_CMD_LINE% -m %BOARD_PKG%\Acpi\BoardAcpiDxe\Dsdt.inf -y 
%PRE_BUILD_REPORT% --log=%PRE_BUILD_LOG%
+echo build %PRE_BUILD_CMD_LINE% -m %BOARD_PKG%\Acpi\BoardAcpiDxe\Dsdt.inf -y 
%PRE_BUILD_REPORT% --log=%PRE_BUILD_LOG% %BINARY_CACHE_CMD_LINE%
+call build %PRE_BUILD_CMD_LINE% -m %BOARD_PKG%\Acpi\BoardAcpiDxe\Dsdt.inf -y 
%PRE_BUILD_REPORT% --log=%PRE_BUILD_LOG% %BINARY_CACHE_CMD_LINE%
 if %ERRORLEVEL% NEQ 0 EXIT /b %ERRORLEVEL%
 
 @REM PSYS == FIX0
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#41700): https://edk2.groups.io/g/devel/message/41700
Mute This Topic: https://groups.io/mt/31875898/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH 3/4] KabylakeOpenBoardPkg:Extend options in bld.bat to support Binary Cache

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1784

Need extend the options in the Kabylake build bld.bat file to
support Binary Cache.

BINARY_CACHE_PATH:
if BINARY_CACHE_PATH is empty, use BinCache as default path

Add "cache-produce" and "cache-consume" to command line,
Used to generate and use Binary Cache files.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Steven Shi 
Cc: Chasel Chiu 
Cc: Michael Kubacki 
Cc: Nate DeSimone 
---
 Platform/Intel/KabylakeOpenBoardPkg/KabylakeRvp3/bld.bat | 23 
+--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/Platform/Intel/KabylakeOpenBoardPkg/KabylakeRvp3/bld.bat 
b/Platform/Intel/KabylakeOpenBoardPkg/KabylakeRvp3/bld.bat
index 449660b75d..af397a3115 100644
--- a/Platform/Intel/KabylakeOpenBoardPkg/KabylakeRvp3/bld.bat
+++ b/Platform/Intel/KabylakeOpenBoardPkg/KabylakeRvp3/bld.bat
@@ -62,6 +62,11 @@ copy /y /b 
%WORKSPACE_FSP_BIN%\KabylakeFspBinPkg\Fsp_Rebased_S.fd+%WORKSPACE_FSP
 @SET REBUILD_MODE=
 @SET BUILD_ROM_ONLY=
 
+@if not defined BINARY_CACHE_PATH (
+  echo Info: BINARY_CACHE_PATH is empty, use BinCache as default
+  SET BINARY_CACHE_PATH=BinCache
+)
+
 :: Loop through arguements until all are processed
 
 :BUILD_FLAGS_LOOP
@@ -87,6 +92,19 @@ copy /y /b 
%WORKSPACE_FSP_BIN%\KabylakeFspBinPkg\Fsp_Rebased_S.fd+%WORKSPACE_FSP
   shift
   goto BUILD_FLAGS_LOOP
 )
+
+@if "%~1" == "cache-produce" (
+  SET BINARY_CACHE_CMD_LINE= --hash --binary-destination=%BINARY_CACHE_PATH%
+  shift
+  goto BUILD_FLAGS_LOOP
+)
+
+@if "%~1" == "cache-consume" (
+  SET BINARY_CACHE_CMD_LINE= --hash --binary-source=%BINARY_CACHE_PATH%
+  shift
+  goto BUILD_FLAGS_LOOP
+)
+
 :: Unknown build flag.
 shift
 goto BUILD_FLAGS_LOOP
@@ -99,11 +117,12 @@ goto BUILD_FLAGS_LOOP
 @echoSILENT_MODE = %SILENT_MODE%
 @echoREBUILD_MODE = %REBUILD_MODE%
 @echoBUILD_ROM_ONLY = %BUILD_ROM_ONLY%
+@echoBINARY_CACHE_CMD_LINE = %BINARY_CACHE_CMD_LINE%
 @echo.
 
 @if %SILENT_MODE% EQU TRUE goto BldSilent
 
-call build -n %NUMBER_OF_PROCESSORS% %REBUILD_MODE% %EXT_BUILD_FLAGS%
+call build -n %NUMBER_OF_PROCESSORS% %REBUILD_MODE% %EXT_BUILD_FLAGS% 
%BINARY_CACHE_CMD_LINE%
 
 @if %ERRORLEVEL% NEQ 0 goto BldFail
 @echo.
@@ -122,7 +141,7 @@ call %WORKSPACE_PLATFORM%\%PROJECT%\postbuild.bat 
%BUILD_ROM_ONLY%
 @echo  
>> Build.log
 @echo. >> Build.log
 
-call build -n %NUMBER_OF_PROCESSORS% %REBUILD_MODE% %EXT_BUILD_FLAGS% 
1>>Build.log 2>&1
+call build -n %NUMBER_OF_PROCESSORS% %REBUILD_MODE% %EXT_BUILD_FLAGS% 
%BINARY_CACHE_CMD_LINE% 1>>Build.log 2>&1
 
 @if %ERRORLEVEL% NEQ 0 goto BldFail
 @echo. >> Build.log
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#41699): https://edk2.groups.io/g/devel/message/41699
Mute This Topic: https://groups.io/mt/31875886/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH 2/4] Platform/Intel:Add build parameter to support Binary Cache

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1784
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1785

Need extend the options in the Intel/build_bios.py file
to support Binary Cache.

 --hash:
   Enable hash-based caching during build process.
 --binary-destination:
   Generate a cache of binary files in the specified directory.
 --binary-source:
   Consume a cache of binary files from the specified directory.

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Steven Shi 
Cc: Shifei A Lu 
Cc: Xiaohu Zhou 
Cc: Isaac W Oram 
Cc: Chasel Chiu 
Cc: Michael Kubacki 
Cc: Nate DeSimone 
Signed-off-by: Zhiju.Fan 
---
 Platform/Intel/build_bios.py | 27 +++
 1 file changed, 27 insertions(+)

diff --git a/Platform/Intel/build_bios.py b/Platform/Intel/build_bios.py
index 9f8d78f6e8..628b127417 100644
--- a/Platform/Intel/build_bios.py
+++ b/Platform/Intel/build_bios.py
@@ -333,6 +333,7 @@ def build(config):
 print(" SILENT_MODE= ", config.get("SILENT_MODE"))
 print(" REBUILD_MODE   = ", config.get("REBUILD_MODE"))
 print(" BUILD_ROM_ONLY = ", config.get("BUILD_ROM_ONLY"))
+print(" BINARY_CACHE_CMD_LINE = ", config.get("HASH"), 
config.get("BINARY_CACHE_CMD_LINE"))
 print("==")
 
 command = ["build", "-n", config["NUMBER_OF_PROCESSORS"]]
@@ -343,6 +344,10 @@ def build(config):
 if config["EXT_BUILD_FLAGS"] and config["EXT_BUILD_FLAGS"] != "":
 command.append(config["EXT_BUILD_FLAGS"])
 
+if config.get('BINARY_CACHE_CMD_LINE'):
+command.append(config['HASH'])
+command.append(config['BINARY_CACHE_CMD_LINE'])
+
 if config.get("SILENT_MODE", "FALSE") == "TRUE":
 command.append("--silent")
 command.append("--quiet")
@@ -848,6 +853,17 @@ def get_cmd_config_arguments(arguments):
 if arguments.fspapi is True:
 result["API_MODE_FSP_WRAPPER_BUILD"] = "TRUE"
 
+if not arguments.UseHashCache:
+result['BINARY_CACHE_CMD_LINE'] = ''
+elif arguments.BinCacheDest:
+result['HASH'] = '--hash'
+result['BINARY_CACHE_CMD_LINE'] = '--binary-destination=%s' % 
arguments.BinCacheDest
+elif arguments.BinCacheSource:
+result['HASH'] = '--hash'
+result['BINARY_CACHE_CMD_LINE'] = '--binary-source=%s' % 
arguments.BinCacheSource
+else:
+result['BINARY_CACHE_CMD_LINE'] = ''
+
 return result
 
 
@@ -924,6 +940,17 @@ def get_cmd_arguments(build_config):
 parser.add_argument("--fspapi", help="API mode fsp wrapper build enabled",
 action='store_true', dest="fspapi")
 
+parser.add_argument("--hash", action="store_true", dest="UseHashCache", 
default=False,
+help="Enable hash-based caching during build process.")
+
+parser.add_argument("--binary-destination", help="Generate a cache of 
binary \
+files in the specified directory.",
+action='store', dest="BinCacheDest")
+
+parser.add_argument("--binary-source", help="Consume a cache of binary 
files \
+from the specified directory.",
+action='store', dest="BinCacheSource")
+
 return parser.parse_args()
 
 
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#41698): https://edk2.groups.io/g/devel/message/41698
Mute This Topic: https://groups.io/mt/31875862/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH 1/4] Intel/Readme.md:Add instructions about Binary Cache in Readme.md

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1784
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1785

Add detailed instructions about Binary Cache in Readme.md,
Extend options to support Binary Cache in the Kabylake
build bld.bat file, Purley build bld.bat file, build_bios.py

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Steven Shi 
Cc: Shifei A Lu 
Cc: Xiaohu Zhou 
Cc: Isaac W Oram 
Cc: Chasel Chiu 
Cc: Michael Kubacki 
Cc: Nate DeSimone 
Signed-off-by: Zhiju.Fan 
---
 Platform/Intel/Readme.md | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/Platform/Intel/Readme.md b/Platform/Intel/Readme.md
index 443fb409b3..41ae99e8e6 100644
--- a/Platform/Intel/Readme.md
+++ b/Platform/Intel/Readme.md
@@ -133,6 +133,9 @@ return back to the minimum platform caller.
   | --silent  | silent build enabled|
   | --performance | performance build enabled   |
   | --fsp | fsp build enabled   |
+  | --hash| Enable hash-based caching   |
+  | --binary-destination  | create cache in specified directory |
+  | --binary-source   | Consume cache from directory|
   | |
 
 * For more information on build options
@@ -191,14 +194,18 @@ For KabylakeOpenBoardPkg
 2. Type "cd edk2-platforms\Platform\Intel\KabylakeOpenBoardPkg\KabylakeRvp3".
 3. Type "GitEdk2MinKabylake.bat" to setup GIT environment.
 4. Type "prep" and make prebuild finish for debug build, "prep r" for release 
build.
-5. Type "bld" to build Kaby Lake reference platform UEFI firmware image.
+5. Type "bld" to build Kaby Lake reference platform UEFI firmware image, "bld 
cache-produce" Generate a cache of binary
+   files in the specified directory, "bld cache-consume" Consume a cache of 
binary files from the specified directory,
+   BINARY_CACHE_PATH is empty, used "BinCache" as default path.

 For PurleyOpenBoardPkg
 1. Open command window, go to the workspace directory, e.g. c:\Purley.
 2. Type "cd edk2-platforms\Platform\Intel\PurleyOpenBoardPkg\BoardMtOlympus".
 3. Type "GitEdk2MinMtOlympus.bat" to setup GIT environment.
 4. Type "bld" to build Purley Mt Olympus board UEFI firmware image, "bld 
release" for release build, "bld clean" to
-   remove intermediate files.
+   remove intermediate files. "bld cache-produce" Generate a cache of binary 
files in the specified directory,
+   "bld cache-consume" Consume a cache of binary files from the specified 
directory, BINARY_CACHE_PATH is empty,
+   used "BinCache" as default path. 
 
 The validated version of iasl compiler that can build MinPurley is 20180629. 
Older version may generate ACPI build
 errors.
-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#41697): https://edk2.groups.io/g/devel/message/41697
Mute This Topic: https://groups.io/mt/31875851/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

[edk2-devel] [PATCH 0/4] Makes binary cache available in Platform/Intel

[Fan, ZhijuX]

Add detailed instructions about Binary Cache in Readme.md,
Extend options to support Binary Cache in the Kabylake
build bld.bat file, Purley build bld.bat file, build_bios.py

Cc: Liming Gao 
Cc: Bob Feng 
Cc: Steven Shi 
Cc: Shifei A Lu 
Cc: Xiaohu Zhou 
Cc: Isaac W Oram 
Cc: Chasel Chiu 
Cc: Michael Kubacki 
Cc: Nate DeSimone 
Signed-off-by: Zhiju.Fan 

Fan, Zhiju (4):
  Intel/Readme.md:Add detailed instructions about Binary Cache in
Readme.md
  Platform/Intel:Add build parameter to support Binary Cache
  KabylakeOpenBoardPkg:Extend options in bld.bat to support Binary Cache
  PurleyOpenBoardPkg:Extend options in bld.bat to support Binary Cache

 Platform/Intel/KabylakeOpenBoardPkg/KabylakeRvp3/bld.bat  | 23 
+--
 Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/bld.bat  | 21 
+++--
 Platform/Intel/PurleyOpenBoardPkg/BoardMtOlympus/prebuild.bat |  4 ++--
 Platform/Intel/Readme.md  | 11 +--
 Platform/Intel/build_bios.py  | 27 
+++
 5 files changed, 78 insertions(+), 8 deletions(-)

-- 
2.14.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#41696): https://edk2.groups.io/g/devel/message/41696
Mute This Topic: https://groups.io/mt/31875841/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

<>

Re: [edk2-devel] [PATCH V5] BaseTools:Make BaseTools support new rules to generate RAW FFS FILE

[Fan, ZhijuX]

The Dict parameter is not used in this function, I will enter an BZ: 
https://bugzilla.tianocore.org/show_bug.cgi?id=1858  to clean up the unused 
parameters



Any question, please let me know. Thanks.

Best Regards
Fan Zhiju



> -Original Message-
> From: Feng, Bob C
> Sent: Wednesday, May 29, 2019 3:16 PM
> To: Fan, ZhijuX ; devel@edk2.groups.io
> Cc: Gao, Liming 
> Subject: RE: [PATCH V5] BaseTools:Make BaseTools support new rules to
> generate RAW FFS FILE
> 
> Hi Zhiju,
> 
> For the changes,
> 
> -def __InfParse__(self, Dict = {}):
> +def __InfParse__(self, Dict = None, IsGenFfs=False):
> 
> and
> 
> -def GetFileList(FfsInf, FileType, FileExtension, Dict = {}, 
> IsMakefile=False):
> +def GetFileList(FfsInf, FileType, FileExtension, Dict = None,
> IsMakefile=False, SectionType=None):
> 
> I think you need to add
> If Dict is None:
> Dict = {}
> 
> In the function body.
> 
> 
> Thanks,
> Bob
> 
> -Original Message-
> From: Fan, ZhijuX
> Sent: Wednesday, May 29, 2019 1:30 PM
> To: devel@edk2.groups.io
> Cc: Gao, Liming ; Feng, Bob C 
> Subject: [PATCH V5] BaseTools:Make BaseTools support new rules to
> generate RAW FFS FILE
> 
> BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1765
> 
> If RAW FFS File Rule has no section for its data.For RAW FFS File, directly 
> call
> GenFfs tool to generate FFS file.
> 
> Ffs Rule:
> [Rule.Common.USER_DEFINED.MicroCode]
>   FILE RAW = $(NAMED_GUID) {
> $(INF_OUTPUT)/$(MODULE_NAME).bin
>   }
> [Rule.Common.USER_DEFINED.LOGO]
>   FILE RAW = $(NAMED_GUID) {
>|.bmp
>   }
> 
> As shown in the rule above,if SectionType and FileType not defined, FFS files
> are generated directly, and no other type of file is generated.
> 
> The patch is to make the BaseTools support these two rules
> 
> Cc: Bob Feng 
> Cc: Liming Gao 
> Signed-off-by: Zhiju.Fan 
> ---
>  BaseTools/Source/Python/Common/DataType.py|  1 +
>  BaseTools/Source/Python/GenFds/EfiSection.py  | 22
> +-
>  BaseTools/Source/Python/GenFds/FdfParser.py   | 14 --
>  BaseTools/Source/Python/GenFds/FfsInfStatement.py |  9 ++---
>  BaseTools/Source/Python/GenFds/Section.py |  7 ++-
>  5 files changed, 46 insertions(+), 7 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/Common/DataType.py
> b/BaseTools/Source/Python/Common/DataType.py
> index 7cd67bc01a..83ec36c235 100644
> --- a/BaseTools/Source/Python/Common/DataType.py
> +++ b/BaseTools/Source/Python/Common/DataType.py
> @@ -122,6 +122,7 @@ BINARY_FILE_TYPE_VER = 'VER'
>  BINARY_FILE_TYPE_UI = 'UI'
>  BINARY_FILE_TYPE_BIN = 'BIN'
>  BINARY_FILE_TYPE_FV = 'FV'
> +BINARY_FILE_TYPE_RAW = 'RAW_BINARY'
> 
>  PLATFORM_COMPONENT_TYPE_LIBRARY_CLASS = 'LIBRARY_CLASS'
>  PLATFORM_COMPONENT_TYPE_MODULE = 'MODULE'
> diff --git a/BaseTools/Source/Python/GenFds/EfiSection.py
> b/BaseTools/Source/Python/GenFds/EfiSection.py
> index 302f244faf..74f176cfef 100644
> --- a/BaseTools/Source/Python/GenFds/EfiSection.py
> +++ b/BaseTools/Source/Python/GenFds/EfiSection.py
> @@ -93,7 +93,7 @@ class EfiSection (EfiSectionClassObject):
>  if '.depex' in SuffixMap:
>  FileList.append(Filename)
>  else:
> -FileList, IsSect = Section.Section.GetFileList(FfsInf, 
> self.FileType,
> self.FileExtension, Dict, IsMakefile=IsMakefile)
> +FileList, IsSect = Section.Section.GetFileList(FfsInf,
> + self.FileType, self.FileExtension, Dict, IsMakefile=IsMakefile,
> + SectionType=SectionType)
>  if IsSect :
>  return FileList, self.Alignment
> 
> @@ -217,6 +217,26 @@ class EfiSection (EfiSectionClassObject):
>   Ui=StringData, 
> IsMakefile=IsMakefile)
>  OutputFileList.append(OutputFile)
> 
> +#
> +# If Section Type is BINARY_FILE_TYPE_RAW
> +#
> +elif SectionType == BINARY_FILE_TYPE_RAW:
> +"""If File List is empty"""
> +if FileList == []:
> +if self.Optional == True:
> +GenFdsGlobalVariable.VerboseLogger("Optional Section 
> don't
> exist!")
> +return [], None
> +else:
> +EdkLogger.error("GenFds", GENFDS_ERROR, "Output
> + file for %s section could not be found for %s" % (SectionType,
> + InfFileName))
> +
> +elif len(FileList) > 1:
> +EdkLogger.error("GenFds", GENFDS_ERROR

[edk2-devel] [PATCH V2] BaseTools:add UniTool.py to Edk2\BaseTools\Scripts

[Fan, ZhijuX]

BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1855

UniTool is one python script to generate UQI (Universal Question
Identifier) unicode string for HII question PROMPT string. UQI
string can be used to identify each HII question.
The scripts function will sync up UQI definitions with uni files
based on vfi/vfr/hfr/sd/sdi in the tree.

This script can be run in both Py2 and Py3.

Cc: Bob Feng 
Cc: Liming Gao 
Signed-off-by: Zhiju.Fan 
---
 BaseTools/Scripts/UniTool.py | 485 +++
 1 file changed, 485 insertions(+)
 create mode 100644 BaseTools/Scripts/UniTool.py

diff --git a/BaseTools/Scripts/UniTool.py b/BaseTools/Scripts/UniTool.py
new file mode 100644
index 00..2f76d305cd
--- /dev/null
+++ b/BaseTools/Scripts/UniTool.py
@@ -0,0 +1,485 @@
+## @file
+# generate UQI (Universal Question Identifier) unicode string for HII question 
PROMPT string. UQI string can be used to
+# identify each HII question.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+
+import re, sys, os, getopt, codecs, fnmatch
+
+# global variable declarations
+questionError = False
+uqiList = re.compile('^#string[ \t]+([A-Z_0-9]+)[ \t]+#language[ \t]+uqi[ 
\t\r\n]+"(?:[x\S]{1,2})([0-9a-fA-F]{4,5})"',re.M).findall
+allUqis = {}
+stringDict = {}
+GlobalVarId = {}
+options = {}
+
+#**
+# description: Prints help information
+#
+# arguments:   none
+#
+# returns: none
+#
+
+def usage () :
+  sys.exit("Syntax:  %s [-b] [-u] [-l] [-x] [-h] [-d 'rootDirectory1'] [-d 
'rootDirectory2'] [-d 'rootDirectory3']... [-q e|w] \
+'rootDirectory0' 'uqiFile'|'uqiFileDirectory' ['excludedDirectory1'] 
['excludedDirectory2'] ['excludedDirectory3']...\n%s" %
+(os.path.basename(sys.argv[0]),
+  """\nFunction will sync up UQI definitions with uni files based on 
vfi/vfr/hfr/sd/sdi in the tree.\n
+Required Arguments:
+  'rootdirectory0'   path to root directory
+  'uqiFileDirectory' path to UQI file(UqiList.uni)
+  'uqiFile'  UQI file
+
+Options:
+  -hShow this help
+  -bBuild option returns error if any new UQI needs 
assigning
+based on vfi/vfr/hfr/sd/sdi when no -u option is 
specified
+  -uCreate new UQIs that does not already exist in uqiFile 
for
+any string requiring a UQI based on vfi/vfr/hfr/sd/sdi
+NOTE: 'uqiFile' cannot be readonly!
+  -lLanguage deletion option (keeps only English and uqi)
+moves all UQIs to 'uqiFile'
+NOTE: Uni files cannot be readonly!
+  -xExclude 'rootDirectory'/'excludedDirectory1' &
+'rootDirectory'/'excludedDirectory2'... from UQI list 
build
+NOTE: Cannot be the same as rootDirectory
+  -dAdd multiple root directories to process
+  -qPrint warning(w) or return error(e) if different HII 
questions
+are referring same string token
+
+Return error if any duplicated UQI string or value in UQI list or if no 
definition
+for any string referred by HII question when -b or -u is specified
+
+NOTE: Options must be specified before parameters
+"""))
+
+#**
+# description: Get uni file encoding
+#
+# arguments:   filename - name of uni file
+#
+# returns: utf-8 or utf-16
+#
+def GetUniFileEncoding(filename):
+  #
+  # Detect Byte Order Mark at beginning of file.  Default to UTF-8
+  #
+  Encoding = 'utf-8'
+
+  #
+  # Read file
+  #
+  try:
+with open(filename, mode='rb') as UniFile:
+  FileIn = UniFile.read()
+  except:
+return Encoding
+
+  if (FileIn.startswith(codecs.BOM_UTF16_BE) or 
FileIn.startswith(codecs.BOM_UTF16_LE)):
+Encoding = 'utf-16'
+
+  return Encoding
+
+# rewrite function os.path.walk
+def Walk(top, func, arg):
+  try:
+names = os.listdir(top)
+  except os.error:
+return
+  func(arg, top, names)
+  for name in names:
+name = os.path.join(top, name)
+if os.path.isdir(name):
+  Walk(name, func, arg)
+
+#**
+# description: Parses commandline arguments and options
+#  Calls function processUni to build dictionary of strings
+#  Calls other functions according to user specified options
+#
+# arguments:   argv - contains all input from command line
+#   - must contain path to root directory
+#   - may contain options -h, -u, -l, -b or -x before path
+#
+# returns: none
+#
+def main(argv) :
+# Read input arguments and options
+  global allUqis, uqiList, questionError
+  try:
+opts, args = getopt.getopt(argv[1:], "hulbxd:q:") # each letter