Re: [edk2] [Patch] BaseTools: Support multi thread build Basetool on Windows

2018-08-29 Thread Gao, Liming
Jaben:
  NmakeSubdirs.bat will be removed once NmakeSubdirs.py is stable. Now, bat is 
kept for a while.

Thanks
Liming
> -Original Message-
> From: Carsey, Jaben
> Sent: Wednesday, August 29, 2018 12:10 AM
> To: Gao, Liming 
> Cc: edk2-devel@lists.01.org; Guo, Dongao 
> Subject: Re: [edk2] [Patch] BaseTools: Support multi thread build Basetool on 
> Windows
> 
> This looks like a great change.  Why not delete the bat file?
> 
> Reviewed-by Jaben Carsey: 
> 
> Jaben
> 
> > On Aug 28, 2018, at 8:28 AM, Liming Gao  wrote:
> >
> > From: Dongao Guo 
> >
> > Add NmakeSubdirs.py to replace NmakeSubdirs.bat in VS Makefile. This script 
> > will
> > invoke nmake in multi thread mode. It can save more than half time of 
> > BaseTools
> > C clean build.
> > GCC make supports multiple thread in make phase. So, GNNmakefile doesn't 
> > need apply
> > this script.
> >
> > single task or job=1:
> >just single thread and invoke subprocess,subprocess will use
> >system.stdout to print output.
> > multi task:
> >thread number is logic cpu count.All subprocess output will pass to
> >python script by PIPE and then script print it to system.stdout.
> >
> > Contributed-under: TianoCore Contribution Agreement 1.1
> > Signed-off-by: Dongao Guo
> > Reviewed-by: Liming Gao 
> > Test-by: Liming Gao 
> > ---
> > BaseTools/Makefile   |  12 +-
> > BaseTools/Source/C/Makefile  |  14 +--
> > BaseTools/Source/C/Makefiles/NmakeSubdirs.py | 169 
> > +++
> > 3 files changed, 182 insertions(+), 13 deletions(-)
> > create mode 100644 BaseTools/Source/C/Makefiles/NmakeSubdirs.py
> >
> > diff --git a/BaseTools/Makefile b/BaseTools/Makefile
> > index 3736d85..b98cd85 100644
> > --- a/BaseTools/Makefile
> > +++ b/BaseTools/Makefile
> > @@ -1,7 +1,7 @@
> > ## @file
> > # Windows makefile for Base Tools project build.
> > #
> > -# Copyright (c) 2007 - 2017, Intel Corporation. All rights reserved.
> > +# Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.
> > # This program and the accompanying materials
> > # are licensed and made available under the terms and conditions of the BSD 
> > License
> > # which accompanies this distribution.  The full text of the license may be 
> > found at
> > @@ -20,19 +20,19 @@ SUBDIRS = $(BASE_TOOLS_PATH)\Source\C 
> > $(BASE_TOOLS_PATH)\Source\Python
> > all: c python
> >
> > c :
> > -  @$(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.bat all 
> > $(BASE_TOOLS_PATH)\Source\C
> > +  @$(PYTHON_HOME)\python.exe 
> > $(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.py  all
> $(BASE_TOOLS_PATH)\Source\C
> >
> > python:
> > -  @$(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.bat all 
> > $(BASE_TOOLS_PATH)\Source\Python
> > +  @$(PYTHON_HOME)\python.exe 
> > $(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.py  all
> $(BASE_TOOLS_PATH)\Source\Python
> >
> > subdirs: $(SUBDIRS)
> > -  @$(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.bat all $**
> > +  @$(PYTHON_HOME)\python.exe 
> > $(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.py  all $**
> >
> > .PHONY: clean
> > clean:
> > -  @$(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.bat clean $(SUBDIRS)
> > +  $(PYTHON_HOME)\python.exe 
> > $(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.py clean $(SUBDIRS)
> >
> > .PHONY: cleanall
> > cleanall:
> > -  @$(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.bat cleanall 
> > $(SUBDIRS)
> > +  $(PYTHON_HOME)\python.exe 
> > $(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.py  cleanall $(SUBDIRS)
> >
> > diff --git a/BaseTools/Source/C/Makefile b/BaseTools/Source/C/Makefile
> > index 5428180..1246d23 100644
> > --- a/BaseTools/Source/C/Makefile
> > +++ b/BaseTools/Source/C/Makefile
> > @@ -1,7 +1,7 @@
> > ## @file
> > # Windows makefile for C tools build.
> > #
> > -# Copyright (c) 2009 - 2017, Intel Corporation. All rights reserved.
> > +# Copyright (c) 2009 - 2018, Intel Corporation. All rights reserved.
> > # This program and the accompanying materials
> > # are licensed and made available under the terms and conditions of the BSD 
> > License
> > # which accompanies this distribution.  The full text of the license may be 
> > found at
> > @@ -16,7 +16,7 @@ HOST_ARCH = IA32
> >
> > LIBRARIES = Common
> > APPLICATIONS = \
> > -  BootSectImage \
> > +  

Re: [edk2] [Patch] BaseTools: Support multi thread build Basetool on Windows

2018-08-28 Thread Carsey, Jaben
This looks like a great change.  Why not delete the bat file?

Reviewed-by Jaben Carsey: 

Jaben

> On Aug 28, 2018, at 8:28 AM, Liming Gao  wrote:
> 
> From: Dongao Guo 
> 
> Add NmakeSubdirs.py to replace NmakeSubdirs.bat in VS Makefile. This script 
> will
> invoke nmake in multi thread mode. It can save more than half time of 
> BaseTools
> C clean build.
> GCC make supports multiple thread in make phase. So, GNNmakefile doesn't need 
> apply
> this script.
> 
> single task or job=1:
>just single thread and invoke subprocess,subprocess will use
>system.stdout to print output.
> multi task:
>thread number is logic cpu count.All subprocess output will pass to
>python script by PIPE and then script print it to system.stdout.
> 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Dongao Guo
> Reviewed-by: Liming Gao 
> Test-by: Liming Gao 
> ---
> BaseTools/Makefile   |  12 +-
> BaseTools/Source/C/Makefile  |  14 +--
> BaseTools/Source/C/Makefiles/NmakeSubdirs.py | 169 +++
> 3 files changed, 182 insertions(+), 13 deletions(-)
> create mode 100644 BaseTools/Source/C/Makefiles/NmakeSubdirs.py
> 
> diff --git a/BaseTools/Makefile b/BaseTools/Makefile
> index 3736d85..b98cd85 100644
> --- a/BaseTools/Makefile
> +++ b/BaseTools/Makefile
> @@ -1,7 +1,7 @@
> ## @file
> # Windows makefile for Base Tools project build.
> #
> -# Copyright (c) 2007 - 2017, Intel Corporation. All rights reserved.
> +# Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.
> # This program and the accompanying materials
> # are licensed and made available under the terms and conditions of the BSD 
> License
> # which accompanies this distribution.  The full text of the license may be 
> found at
> @@ -20,19 +20,19 @@ SUBDIRS = $(BASE_TOOLS_PATH)\Source\C 
> $(BASE_TOOLS_PATH)\Source\Python
> all: c python
> 
> c :
> -  @$(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.bat all 
> $(BASE_TOOLS_PATH)\Source\C
> +  @$(PYTHON_HOME)\python.exe 
> $(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.py  all 
> $(BASE_TOOLS_PATH)\Source\C
> 
> python:
> -  @$(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.bat all 
> $(BASE_TOOLS_PATH)\Source\Python
> +  @$(PYTHON_HOME)\python.exe 
> $(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.py  all 
> $(BASE_TOOLS_PATH)\Source\Python
> 
> subdirs: $(SUBDIRS)
> -  @$(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.bat all $**
> +  @$(PYTHON_HOME)\python.exe 
> $(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.py  all $**
> 
> .PHONY: clean
> clean:
> -  @$(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.bat clean $(SUBDIRS)
> +  $(PYTHON_HOME)\python.exe 
> $(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.py clean $(SUBDIRS)
> 
> .PHONY: cleanall
> cleanall:
> -  @$(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.bat cleanall $(SUBDIRS)
> +  $(PYTHON_HOME)\python.exe 
> $(BASE_TOOLS_PATH)\Source\C\Makefiles\NmakeSubdirs.py  cleanall $(SUBDIRS)
> 
> diff --git a/BaseTools/Source/C/Makefile b/BaseTools/Source/C/Makefile
> index 5428180..1246d23 100644
> --- a/BaseTools/Source/C/Makefile
> +++ b/BaseTools/Source/C/Makefile
> @@ -1,7 +1,7 @@
> ## @file
> # Windows makefile for C tools build.
> #
> -# Copyright (c) 2009 - 2017, Intel Corporation. All rights reserved.
> +# Copyright (c) 2009 - 2018, Intel Corporation. All rights reserved.
> # This program and the accompanying materials
> # are licensed and made available under the terms and conditions of the BSD 
> License
> # which accompanies this distribution.  The full text of the license may be 
> found at
> @@ -16,7 +16,7 @@ HOST_ARCH = IA32
> 
> LIBRARIES = Common
> APPLICATIONS = \
> -  BootSectImage \
> +  VfrCompile \
>   BrotliCompress \
>   EfiLdrImage \
>   EfiRom \
> @@ -32,7 +32,7 @@ APPLICATIONS = \
>   Split \
>   TianoCompress \
>   VolInfo \
> -  VfrCompile \
> +  BootSectImage \
>   DevicePath
> 
> all: libs apps install
> @@ -43,7 +43,7 @@ libs: $(LIBRARIES)
>@echo # Build libraries
>@echo ##
>@if not exist $(LIB_PATH) mkdir $(LIB_PATH)
> -@Makefiles\NmakeSubdirs.bat all $**
> +@$(PYTHON_HOME)\python.exe Makefiles\NmakeSubdirs.py all $**
> 
> apps: $(APPLICATIONS)
>@echo.
> @@ -51,7 +51,7 @@ apps: $(APPLICATIONS)
>@echo # Build executables
>@echo ##
>@if not exist $(BIN_PATH) mkdir $(BIN_PATH)
> -@Makefiles\NmakeSubdirs.bat all $**
> +@$(PYTHON_HOME)\python.exe Makefiles\NmakeSubdirs.py all $**
> 
> install: $(LIB_PATH) $(BIN_PATH)
>@echo.
> @@ -65,11 +65,11 @@ install: $(LIB_PATH) $(BIN_PATH)
> 
> .PHONY: clean
> clean:
> -  @Makefiles\NmakeSubdirs.bat clean $(LIBRARIES) $(APPLICATIONS)
> +  @$(PYTHON_HOME)\python.exe Makefiles\NmakeSubdirs.py clean $(LIBRARIES) 
> $(APPLICATIONS)
> 
> .PHONY: cleanall
> cleanall:
> -  @Makefiles\NmakeSubdirs.bat cleanall $(LIBRARIES) $(APPLICATIONS)
> +  @$(PYTHON_HOME)\python.exe