Re: [edk2-devel] BaseTools --hash malfunction after migrate from stable201905 to stable201908

2019-09-28 Thread Steven Shi
Hi Derek, Thank you for the data. I will continue to enhance the –hash performance, and will let you know after I figure out new solution. Thanks Steven Shi Intel\SSG\SFE\FIE Firmware Infrastructure From: Lin, Derek (HPS SW) [mailto:derek.l...@hpe.com] Sent: Friday, September 27, 2019 3:57

Re: [edk2-devel] BaseTools --hash malfunction after migrate from stable201905 to stable201908

2019-09-24 Thread Steven Shi
. Thanks Steven Shi Intel\SSG\FID\Firmware Infrastructure From: Shi, Steven Sent: Monday, September 23, 2019 3:52 PM To: 'Lin, Derek (HPS SW)' ; devel@edk2.groups.io Cc: Feng, Bob C ; Gao, Liming (liming@intel.com) Subject: RE: BaseTools --hash malfunction after migrate from stable201905

Re: [edk2-devel] BaseTools --hash malfunction after migrate from stable201905 to stable201908

2019-09-23 Thread Steven Shi
Hi Derek, Thank you to raise this issue. We will fix it and add back the --hash original functionality. Thanks Steven From: Lin, Derek (HPS SW) [mailto:derek.l...@hpe.com] Sent: Friday, September 20, 2019 2:54 PM To: devel@edk2.groups.io Cc: Feng, Bob C ; Shi, Steven Subject: BaseTools --hash

[edk2-devel] [PATCH v2 0/1] BaseTools: Add more parameter checking for CopyFileOnChange()

2019-09-17 Thread Steven Shi
This patch is to add directory input parameter type checking and error message output for method CopyFileOnChange. V2: Remove the redundant os.path.exists checking for better performance V1: Initial patch Steven Shi (1): BaseTools: Add more parameter checking for CopyFileOnChange

[edk2-devel] [PATCH v2 1/1] BaseTools: Add more parameter checking for CopyFileOnChange()

2019-09-17 Thread Steven Shi
for method CopyFileOnChange. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/Common/Misc.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/BaseTools/Source/Python/Common/Misc.py b/BaseTools/Source/Python/Common/Misc.py index 714eb840ea

[edk2-devel] [PATCH] BaseTools: Add more parameter checking for CopyFileOnChange()

2019-09-15 Thread Steven Shi
for method CopyFileOnChange. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/Common/Misc.py | 5 + 1 file changed, 5 insertions(+) diff --git a/BaseTools/Source/Python/Common/Misc.py b/BaseTools/Source/Python/Common/Misc.py index 15ae6a9e40..7c93fc4dca

Re: [edk2-devel] commit "BaseTools: Improve the cache hit in the edk2 build cache" causes build.py unknown fatal error

2019-09-09 Thread Steven Shi
--binary-destination=BinCache · Remove the Build folder $ rm Build/ -r –f · Consume the build cache: $ build -p OvmfPkg/OvmfPkgIa32.dsc -a IA32 -t GCC5 -b DEBUG --hash --binary-source=BinCache Thanks Steven Shi Intel\SSG\FID\Firmware Infrastructure From: Vladimir Olovyannikov

Re: [edk2-devel] commit "BaseTools: Improve the cache hit in the edk2 build cache" causes build.py unknown fatal error

2019-09-06 Thread Steven Shi
Hi Vladimir, Did you enable the build cache? Could you let me know your build steps and commands? Thanks Steven From: Vladimir Olovyannikov [mailto:vladimir.olovyanni...@broadcom.com] Sent: Saturday, September 7, 2019 4:45 AM To: devel@edk2.groups.io; Shi, Steven Cc: Gao, Liming ; Feng, Bob C

[edk2-devel] [PATCH] BaseTools: Support more file types in build cache

2019-08-26 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1958 Current build cache does not store and restore all types file of Hii/vfr, version and dpx. This patch adds more file types to support them in build cache. Cc: Liming Gao Cc: Bob Feng Signed-off-by:

[edk2-devel] [PATCH] [edk2-stable201908] BaseTools: Support long file path in windows for misc functions

2019-08-21 Thread Steven Shi
y and save in windows. This patch enhances them to support the long file path copy and save correctly. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/Common/Misc.py | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/BaseTools/Sou

[edk2-devel] [PATCH] BaseTools: Support long file path in windows for misc functions

2019-08-21 Thread Steven Shi
y and save in windows. This patch enhances them to support the long file path copy and save correctly. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/Common/Misc.py | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/BaseTools/Sou

[edk2-devel] [PATCH v6 4/5] BaseTools: Add GenFds multi-thread support in build cache

2019-08-15 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1923 Fix the issue that the GenFds multi-thread will build fail if enable the build cache together. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/ModuleAutoG

[edk2-devel] [PATCH v6 3/5] BaseTools: Change the [Arch][Name] module key in Build cache

2019-08-15 Thread Steven Shi
j __hash__ and __repr__ attributes already contain the full path and arch name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/GenMake.py | 6 +- BaseTools/Source/Python/build/build.py | 48 ---

[edk2-devel] [PATCH v6 5/5] BaseTools: Improve the file saving and copying reliability

2019-08-15 Thread Steven Shi
ge() might be invoked in multiple sub-processes simultaneously, and this patch adds global locks to sync these functions invoking which can harden their reliability. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGenWorker.py | 6 -- BaseTo

[edk2-devel] [PATCH v6 1/5] BaseTools: Improve the cache hit in the edk2 build cache

2019-08-15 Thread Steven Shi
leverages the module makefile to achieve more accurate and precise dependency info for a module. When the build cache miss with the first quick hash, the Basetool will caculate new one after makefile is generated and then check again. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTo

[edk2-devel] [PATCH v6 2/5] BaseTools: Print first cache missing file for build cachle

2019-08-15 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1925 When a module build cache miss, add support to print the first cache missing file path and name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGenWork

[edk2-devel] [PATCH v6 0/5] Build cache enhancement

2019-08-15 Thread Steven Shi
From: "Shi, Steven" This patch set is for the 201908 stable tag Enhance the edk2 build cache with below patches: Patch 01/05: Improve the cache hit rate through new cache checkpoint and hash algorithm Patch 02/05: Print more info to explain why a module build cache miss Patch 03/05: Fix the

[edk2-devel] [PATCH v5 2/5] BaseTools: Print first cache missing file for build cachle

2019-08-14 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1925 When a module build cache miss, add support to print the first cache missing file path and name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGenWork

[edk2-devel] [PATCH v5 4/5] BaseTools: Add GenFds multi-thread support in build cache

2019-08-14 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1923 Fix the issue that the GenFds multi-thread will build fail if enable the build cache together. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/ModuleAutoG

[edk2-devel] [PATCH v5 0/5] Build cache enhancement

2019-08-14 Thread Steven Shi
From: "Shi, Steven" This patch set is for the 201908 stable tag Enhance the edk2 build cache with below patches: Patch 01/05: Improve the cache hit rate through new cache checkpoint and hash algorithm Patch 02/05: Print more info to explain why a module build cache miss Patch 03/05: Fix the

[edk2-devel] [PATCH v5 1/5] BaseTools: Improve the cache hit in the edk2 build cache

2019-08-14 Thread Steven Shi
leverages the module makefile to achieve more accurate and precise dependency info for a module. When the build cache miss with the first quick hash, the Basetool will caculate new one after makefile is generated and then check again. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTo

[edk2-devel] [PATCH v5 3/5] BaseTools: Change the [Arch][Name] module key in Build cache

2019-08-14 Thread Steven Shi
j __hash__ and __repr__ attributes already contain the full path and arch name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/GenMake.py | 6 +- BaseTools/Source/Python/build/build.py | 48 ---

[edk2-devel] [PATCH v5 5/5] BaseTools: Improve the file saving and copying reliability

2019-08-14 Thread Steven Shi
ge() might be invoked in multiple sub-processes simultaneously, and this patch adds global locks to sync these functions invoking which can harden their reliability. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGenWorker.py | 4 +++- BaseTo

[edk2-devel] [PATCH v4 4/5] BaseTools: Add GenFds multi-thread support in build cache

2019-08-14 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1923 Fix the issue that the GenFds multi-thread will build fail if enable the build cache together. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/ModuleAutoG

[edk2-devel] [PATCH v4 2/5] BaseTools: Print first cache missing file for build cachle

2019-08-14 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1925 When a module build cache miss, add support to print the first cache missing file path and name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGenWork

[edk2-devel] [PATCH v4 3/5] BaseTools: Change the [Arch][Name] module key in Build cache

2019-08-14 Thread Steven Shi
j __hash__ and __repr__ attributes already contain the full path and arch name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/GenMake.py | 6 +- BaseTools/Source/Python/build/build.py | 48 ---

[edk2-devel] [PATCH v4 1/5] BaseTools: Improve the cache hit in the edk2 build cache

2019-08-14 Thread Steven Shi
leverages the module makefile to achieve more accurate and precise dependency info for a module. When the build cache miss with the first quick hash, the Basetool will caculate new one after makefile is generated and then check again. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTo

[edk2-devel] [PATCH v4 5/5] BaseTools: Improve the file saving and copying reliability

2019-08-14 Thread Steven Shi
ge() might be invoked in multiple sub-processes simultaneously, and this patch adds a global lock to sync these functions invoking which can harden their reliability. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGenWorker.py | 4 +++- BaseTo

[edk2-devel] [PATCH v4 0/5] Build cache enhancement

2019-08-14 Thread Steven Shi
From: "Shi, Steven" Enhance the edk2 build cache with below patches: Patch 01/05: Improve the cache hit rate through new cache checkpoint and hash algorithm Patch 02/05: Print more info to explain why a module build cache miss Patch 03/05: Fix the unsafe [self.Arch][self.Name] key usage in

[edk2-devel] [PATCH v3 0/5] Build cache enhancement

2019-08-13 Thread Steven Shi
From: "Shi, Steven" Enhance the edk2 build cache with below patches: Patch 01/05: Improve the cache hit rate through new cache checkpoint and hash algorithm Patch 02/05: Print more info to explain why a module build cache miss Patch 03/05: Fix the unsafe [self.Arch][self.Name] key usage in

[edk2-devel] [PATCH v3 4/5] BaseTools: Add GenFds multi-thread support in build cache

2019-08-13 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1923 Fix the issue that the GenFds multi-thread will build fail if enable the build cache together. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/ModuleAutoG

[edk2-devel] [PATCH v3 2/5] BaseTools: Print first cache missing file for build cachle

2019-08-13 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1925 When a module build cache miss, add support to print the first cache missing file path and name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGenWork

[edk2-devel] [PATCH v3 5/5] BaseTools: Improve the file saving and copying reliability

2019-08-13 Thread Steven Shi
ge() might be invoked in multiple sub-processes simultaneously, and this patch adds a global lock to sync these functions invoking which can harden their reliability. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/GenC.py | 2 +- BaseTools/Sou

[edk2-devel] [PATCH v3 3/5] BaseTools: Change the [Arch][Name] module key in Build cache

2019-08-13 Thread Steven Shi
j __hash__ and __repr__ attributes already contain the full path and arch name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/GenMake.py | 6 +- BaseTools/Source/Python/build/build.py | 49 +---

[edk2-devel] [PATCH v3 1/5] BaseTools: Improve the cache hit in the edk2 build cache

2019-08-13 Thread Steven Shi
leverages the module makefile to achieve more accurate and precise dependency info for a module. When the build cache miss with the first quick hash, the Basetool will caculate new one after makefile is generated and then check again. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTo

[edk2-devel] [PATCH v2 3/4] BaseTools: Change the [Arch][Name] module key in Build cache

2019-08-12 Thread Steven Shi
j __hash__ and __repr__ attributes already contain the full path and arch name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/GenMake.py | 6 +- BaseTools/Source/Python/build/build.py | 49 +---

[edk2-devel] [PATCH v2 4/4] BaseTools: Add GenFds multi-thread support in build cache

2019-08-12 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1923 Fix the issue that the GenFds multi-thread will build fail if enable the build cache together. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/ModuleAutoG

[edk2-devel] [PATCH v2 2/4] BaseTools: Print first cache missing file for build cachle

2019-08-12 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1925 When a module build cache miss, add support to print the first cache missing file path and name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGenWork

[edk2-devel] [PATCH v2 0/4] Build cache enhancement

2019-08-12 Thread Steven Shi
From: "Shi, Steven" Enhance the edk2 build cache with below patches: Patch 01/04: Improve the cache hit rate through new cache checkpoint and hash algorithm Patch 02/04: Print more info to explain why a module build cache miss Patch 03/04: Fix the unsafe [self.Arch][self.Name] key usage in

[edk2-devel] [PATCH v2 1/4] BaseTools: Improve the cache hit in the edk2 build cache

2019-08-12 Thread Steven Shi
leverages the module makefile to achieve more accurate and precise dependency info for a module. When the build cache miss with the first quick hash, the Basetool will caculate new one after makefile is generated and then check again. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTo

[edk2-devel] [PATCH 4/4] BaseTools: Add GenFds multi-thread support in build cache

2019-08-08 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1923 Fix the issue that the GenFds multi-thread will build fail if enable the build cache together. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/ModuleAutoG

[edk2-devel] [PATCH 3/4] BaseTools: Change the [Arch][Name] module key in Build cache

2019-08-08 Thread Steven Shi
j __hash__ and __repr__ attributes already contain the full path and arch name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/GenMake.py | 6 +- BaseTools/Source/Python/build/build.py | 49 +---

[edk2-devel] [PATCH 1/4] BaseTools: Improve the cache hit in the edk2 build cache

2019-08-08 Thread Steven Shi
leverages the module makefile to achieve more accurate and precise dependency info for a module. When the build cache miss with the first quick hash, the Basetool will caculate new one after makefile is generated and then check again. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTo

[edk2-devel] [PATCH 2/4] BaseTools: Print first cache missing file for build cachle

2019-08-08 Thread Steven Shi
From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1925 When a module build cache miss, add support to print the first cache missing file path and name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGenWork

[edk2-devel] [PATCH 0/4] Build cache enhancement

2019-08-08 Thread Steven Shi
From: "Shi, Steven" Enhance the edk2 build cache with below patches: Patch 01/04: Improve the cache hit rate through new cache checkpoint and hash algorithm Patch 02/04: Print more info to explain why a module build cache miss Patch 03/04: Fix the unsafe [self.Arch][self.Name] key usage in

Re: [edk2-devel] [Patch 0/9] Enable multiple process AutoGen

2019-07-21 Thread Steven Shi
is multi-process build tool fails to re-generate the new autogen code (CreateCodeFile) and makefile (CreateMakeFile). The new build tool wrongly skip these steps in incremental build. Thanks Steven Shi > -Original Message- > From: devel@edk2.groups.io [mailto:devel@edk2.grou

Re: [edk2-devel] [Patch 2/2] BaseTools: Move Build Cache related function out of CreateAsBuiltInf

2019-06-24 Thread Steven Shi
Reviewed-by: Steven Shi > -Original Message- > From: Feng, Bob C > Sent: Monday, June 24, 2019 9:52 AM > To: devel@edk2.groups.io > Cc: Feng, Bob C ; Shi, Steven ; > Gao, Liming ; Rodriguez, Christian > > Subject: [Patch 2/2] BaseTools: Move Build Ca

Re: [edk2-devel] [Patch 1/2] BaseTools: refine CreateAsBuiltInf function

2019-06-24 Thread Steven Shi
Reviewed-by: Steven Shi > -Original Message- > From: Feng, Bob C > Sent: Monday, June 24, 2019 9:51 AM > To: devel@edk2.groups.io > Cc: Gao, Liming ; Shi, Steven ; > Rodriguez, Christian ; Feng, Bob C > > Subject: [Patch 1/2] BaseTools: refine CreateAsBuiltInf

[edk2-devel] [PATCH] BaseTools:Change build cache option name from --binary- to --cache-

2019-06-17 Thread Steven Shi
https://bugzilla.tianocore.org/show_bug.cgi?id=1896 Current build cache is enabled by the below two options: --binary-destination, which is to produce the cache. --binary-source, which is to consume the cache. Since current build cache does not only support to store the binary file (e.g.

[edk2-devel] [PATCH v2 1/1] BaseTools:Introduce CopyFileOnChange() function to copy cache files

2019-06-17 Thread Steven Shi
. This patch introduces an atomic copy function to avoid duplicated cache files copy. Cc: Liming Gao Cc: Bob Feng Cc: Christian Rodriguez Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGen.py | 10 +- BaseTools/Source/Python/Common/LongFilePathOs.py | 4

[edk2-devel] [PATCH v2 0/1] BaseTools:Introduce CopyFileOnChange() function to copy cache files

2019-06-17 Thread Steven Shi
V2: set shallow=False in filecmp.cmp() to compare the src and dsc files content use shutil.copy() replace the shutil.copyfile() to do the file copy, which is better to reserve the dsc file metadata V1: Initial fix Steven Shi (1): BaseTools:Introduce CopyFileOnChange() function to copy cache

[edk2-devel] [PATCH v3 0/1] BaseTools: Cannot store library cache of different arch together

2019-06-17 Thread Steven Shi
suggestion, Still keep the set() usage, but instead to enahnce the AutoGen object __hash__ definition to be unique by including the AutoGen object arch string into the __hash_ values. V1: Avoid to use the set() as the container to save the library and module objects Steven Shi (1): BaseTools

[edk2-devel] [PATCH v3 1/1] BaseTools: Cannot store library cache of different arch together

2019-06-17 Thread Steven Shi
: Liming Gao Cc: Bob Feng Cc: Christian Rodriguez Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGen.py | 21 + 1 file changed, 21 insertions(+) diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py index

Re: [edk2-devel] [PATCH] BaseTools: Cannot store library cache of different arch together

2019-06-14 Thread Steven Shi
Hi Christian, I think about BZ 1895 again, and your suggestion of "change the hash to include the arch" is better. I've sent the v2 patch to enhance it. BTW, the CopyFileOnChange() is still necessary for BZ 1894. Thanks Steven Shi Intel\SSG\FID\Firmware Infrastructure From:

[edk2-devel] [PATCH v2 0/1] BaseTools: Cannot store library cache of different arch together

2019-06-14 Thread Steven Shi
Steven Shi (1): BaseTools: Cannot store library cache of different arch together BaseTools/Source/Python/AutoGen/AutoGen.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.17.1.windows.2 -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply

[edk2-devel] [PATCH v2 1/1] BaseTools: Cannot store library cache of different arch together

2019-06-14 Thread Steven Shi
: Christian Rodriguez Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGen.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py index 3f41fbb507..1fe8312d22 100644 --- a/BaseTools

[edk2-devel] [PATCH] BaseTools:Introduce CopyFileOnChange() function to copy cache files

2019-06-14 Thread Steven Shi
. This patch introduces an atomic copy function to avoid duplicated cache files copy. Cc: Liming Gao Cc: Bob Feng Cc: Christian Rodriguez Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGen.py | 11 ++- BaseTools/Source/Python/Common/LongFilePathOs.py | 4

[edk2-devel] [PATCH v2 0/1] BaseTools:Build Cache output notification message

2019-06-14 Thread Steven Shi
V2: Only output message for the driver module and not for library, which can filter duplicated library messages from different driver modules, and make the cache messages look more concise. V1: Initial fix from Zhiju Steven Shi (1): BaseTools:Build Cache output notification message

[edk2-devel] [PATCH v2 1/1] BaseTools:Build Cache output notification message

2019-06-14 Thread Steven Shi
message about the cache miss or hit when build consumes the cache. Cc: Liming Gao Cc: Bob Feng Cc: Zhiju.Fan Signed-off-by: Steven Shi --- BaseTools/Source/Python/build/build.py | 10 ++ 1 file changed, 10 insertions(+) diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools

Re: [edk2-devel] [PATCH] BaseTools: Cannot store library cache of different arch together

2019-06-11 Thread Steven Shi
Sorry, the CopyFileOnChange() will ensure only once IO store/restore writing for each library. The extra IO read is ok. Thanks Steven Shi Intel\SSG\FID\Firmware Infrastructure From: Shi, Steven Sent: Wednesday, June 12, 2019 11:24 AM To: Rodriguez, Christian ; devel@edk2.groups.io Cc: Gao

Re: [edk2-devel] [PATCH] BaseTools: Cannot store library cache of different arch together

2019-06-11 Thread Steven Shi
h avoid to use the set() as the container to save the library and > >module objects because the objects might have same __hash__ value. > > > >Cc: Liming Gao mailto:liming@intel.com>> > >Cc: Bob Feng mailto:bob.c.f...@intel.com>> > >Cc: Christian R

[edk2-devel] [PATCH v2 1/1] BaseTools:Build cache cannot store the cache files for library package

2019-06-11 Thread Steven Shi
build cache files (e.g. .hash and .lib) are stored in the BinCache folder. But the BinCache folder is not generated at all. This patch is going to fix that issue. Cc: Liming Gao Cc: Bob Feng Cc: Zhiju.Fan Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGen.py | 5 - 1

[edk2-devel] [PATCH v2 0/1] BaseTools:Build cache cannot store the cache files for library package

2019-06-11 Thread Steven Shi
V2: As Bob suggestion to remove the "pass" in except block. Enhance the comments as well V1: Initial fix from Zhiju Steven Shi (1): BaseTools:Build cache cannot store the cache files for library package BaseTools/Source/Python/AutoGen/AutoGen.py | 5 - 1 file changed, 4 insert

[edk2-devel] [PATCH] BaseTools: Cannot store library cache of different arch together

2019-06-10 Thread Steven Shi
Rodriguez Signed-off-by: Steven Shi --- BaseTools/Source/Python/build/build.py | 13 +++-- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Python/build/build.py index 0855d4561c..f7a79cbbab 100644 --- a/BaseTools/Source

Re: [edk2-devel] [PATCH v3 1/1] BaseTools:Extend the binary cache to support library cache

2019-05-30 Thread Steven Shi
Yes. The "entend" should be "extend", please help to fix it when pushing. Thanks! Steven Shi > -Original Message- > From: Feng, Bob C > Sent: Thursday, May 30, 2019 10:44 AM > To: devel@edk2.groups.io; Shi, Steven > Cc: Gao, Liming ; Rodriguez, Chris

[edk2-devel] [PATCH v3 1/1] BaseTools:Extend the binary cache to support library cache

2019-05-29 Thread Steven Shi
library. Cc: Liming Gao Cc: Bob Feng Cc: Christian Rodriguez Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGen.py | 6 ++ 1 file changed, 6 insertions(+) diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py index

[edk2-devel] [PATCH v3 0/1] BaseTools:Extend the binary cache to support library

2019-05-29 Thread Steven Shi
Steven Shi (1): BaseTools:Extend the binary cache to support library cache BaseTools/Source/Python/AutoGen/AutoGen.py | 6 ++ 1 file changed, 6 insertions(+) -- 2.17.1.windows.2 -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#41654

Re: [edk2-devel] [PATCH v2] BaseTools:Extend the binary cache to support library cache

2019-05-29 Thread Steven Shi
OK, will move the change notices from commit message to cover letter. Thanks Steven Shi > -Original Message- > From: Leif Lindholm [mailto:leif.lindh...@linaro.org] > Sent: Wednesday, May 29, 2019 7:14 PM > To: devel@edk2.groups.io; Shi, Steven > Cc: Gao, Limin

[edk2-devel] [PATCH v2] BaseTools:Extend the binary cache to support library cache

2019-05-28 Thread Steven Shi
module. If a driver module cache miss happen, all its dependency library modules need rebuild which is very time-consuming. This patch is to entend the binary cache to support library. Cc: Liming Gao Cc: Bob Feng Cc: Christian Rodriguez Signed-off-by: Steven Shi --- BaseTools/Source/Python

Re: [edk2-devel] [PATCH] BaseTools:Extend the binary cache to support library cache

2019-05-28 Thread Steven Shi
OK, will send the new version patch to simplify it. Thanks Steven Shi Intel\SSG\FID\Firmware Infrastructure > -Original Message- > From: Rodriguez, Christian > Sent: Tuesday, May 28, 2019 11:16 PM > To: devel@edk2.groups.io; Shi, Steven > Cc: Gao, Liming ; Feng, Bob C

Re: [edk2-devel] [PATCH] BaseTools:Fix the library dependency missing in Binary Cache

2019-05-28 Thread Steven Shi
OK, will solve this issue through the fix patch of BZ 1797. Thanks Steven Shi Intel\SSG\FID\Firmware Infrastructure > -Original Message- > From: Rodriguez, Christian > Sent: Tuesday, May 28, 2019 11:06 PM > To: devel@edk2.groups.io; Shi, Steven > Cc: Gao, Limin

[edk2-devel] [PATCH] BaseTools:Extend the binary cache to support library cache

2019-05-28 Thread Steven Shi
library. Cc: Liming Gao Cc: Bob Feng Cc: Christian Rodriguez Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++--- BaseTools/Source/Python/build/build.py | 3 ++- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/BaseTools/Source

[edk2-devel] [PATCH] BaseTools:Update binary cache restore time to current time

2019-05-28 Thread Steven Shi
on the time stamp. Enhance to update the restored file creation time to current time. Cc: Liming Gao Cc: Bob Feng Cc: Zhiju Fan Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGen.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/BaseTools/Source

[edk2-devel] [PATCH] BaseTools:Fix the library dependency missing in Binary Cache

2019-05-28 Thread Steven Shi
: Liming Gao Cc: Bob Feng Cc: Christian Rodriguez Signed-off-by: Steven Shi --- BaseTools/Source/Python/build/build.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Python/build/build.py index 80ceb98310..673a9379ba

Re: [edk2-devel] [PATCH] BaseTools: Skip .mcb file module in Binary Cache

2019-05-08 Thread Steven Shi
ild Cache directly. Thanks Steven Shi > -Original Message- > From: Gao, Liming > Sent: Thursday, May 9, 2019 1:47 PM > To: Yao, Jiewen ; devel@edk2.groups.io; Shi, Steven > > Cc: Feng, Bob C ; Rodriguez, Christian > ; Johnson, Michael > > Subject: RE: [edk2-deve

[edk2-devel] [PATCH] BaseTools: Skip .mcb file module in Binary Cache

2019-05-08 Thread Steven Shi
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1723 Current Kabylake open platform will build fail if enabled to consume the binary cache, because the binary cache doesn't support to recovery the .mcb microcode file, e.g. m80406E8_0026.mcb, in a platform level folder which is outside of

Re: [edk2-devel] [PATCH 0/2] Add batch option to support build Binary Cache

2019-05-08 Thread Steven Shi
as default + SET BINARY_CACHE_PATH=BinCache +) + Thanks Steven Shi > -Original Message- > From: Gao, Liming > Sent: Thursday, May 9, 2019 11:37 AM > To: Shi, Steven ; devel@edk2.groups.io > Cc: Chiu, Chasel ; Lu, Shifei A > ; > Bi, Dandan ; Kubacki, Michael A > > S

[edk2-devel] [PATCH 2/2] PurleyOpenBoardPkg/BoardMtOlympus: Extend build to support Binary Cache

2019-05-08 Thread Steven Shi
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=1785 Extend the options in the Purley build batch file to support Binary Cache produce and consume switch. --- .../PurleyOpenBoardPkg/BoardMtOlympus/bld.bat | 18 -- .../BoardMtOlympus/prebuild.bat| 4 ++-- 2

[edk2-devel] [PATCH 1/2] KabylakeOpenBoardPkg/KabylakeRvp3: Extend build to support Binary Cache

2019-05-08 Thread Steven Shi
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=1784 Need extend the options in the Kabylake build bld.bat file to support Binary Cache. --- .../KabylakeOpenBoardPkg/KabylakeRvp3/bld.bat | 24 +-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git

[edk2-devel] [PATCH 0/2] Add batch option to support build Binary Cache

2019-05-08 Thread Steven Shi
This patch serial is to extend the options in the Kabylake and Purley build batch file to support build Binary Cache produce and consume switch. Steven Shi (2): KabylakeOpenBoardPkg/KabylakeRvp3: Extend build to support Binary Cache PurleyOpenBoardPkg/BoardMtOlympus: Extend build

Re: [edk2-devel] [Patch 0/7] Add new CLANG8ELF tool chain for new LLVM/CLANG8

2019-04-30 Thread Steven Shi
equirements. See the background detail here: http://lists.llvm.org/pipermail/llvm-dev/2019-January/129749.html Thanks Steven Shi Intel\SSG\FID\Firmware Infrastructure > -Original Message- > From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Leif > Lindholm > S