Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux

2020-04-12 Thread Liming Gao 
Reviewed-by: Liming Gao 

> -Original Message-
> From: devel@edk2.groups.io  On Behalf Of Michael Kubacki
> Sent: Friday, April 3, 2020 1:05 AM
> To: devel@edk2.groups.io
> Cc: Feng, Bob C ; Gao, Liming 
> Subject: [edk2-devel] [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to 
> use multiple threads on Linux
> 
> From: Sean Brogan 
> 
> REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2640
> 
> Azure Pipelines agents have 2 threads. This commit has been shown to
> reduce the build time in half on those agents.
> 
> Cc: Bob C Feng 
> Cc: Liming Gao 
> Signed-off-by: Michael Kubacki 
> ---
>  BaseTools/Edk2ToolsBuild.py | 16 +++-
>  1 file changed, 15 insertions(+), 1 deletion(-)
> 
> diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py
> index 057d2e9e0633..1ea8187de693 100644
> --- a/BaseTools/Edk2ToolsBuild.py
> +++ b/BaseTools/Edk2ToolsBuild.py
> @@ -11,6 +11,7 @@ import os
>  import sys
>  import logging
>  import argparse
> +import multiprocessing
>  from edk2toolext import edk2_logging
>  from edk2toolext.environment import self_describing_environment
>  from edk2toolext.base_abstract_invocable import BaseAbstractInvocable
> @@ -141,7 +142,8 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
>  return ret
> 
>  elif self.tool_chain_tag.lower().startswith("gcc"):
> -ret = RunCmd("make", "-C .", 
> workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
> +cpu_count = self.GetCpuThreads()
> +ret = RunCmd("make", f"-C .  -j {cpu_count}", 
> workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
>  if ret != 0:
>  raise Exception("Failed to build.")
> 
> @@ -154,6 +156,18 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
>  logging.critical("Tool Chain not supported")
>  return -1
> 
> +def GetCpuThreads(self) -> int:
> +''' Function to return number of cpus. If error return 1'''
> +cpus = 1
> +try:
> +cpus = multiprocessing.cpu_count()
> +except:
> +# from the internet there are cases where cpu_count is not 
> implemented.
> +# will handle error by just doing single proc build
> +pass
> +return cpus
> +
> +
> 
>  def main():
>  Edk2ToolsBuild().Invoke()
> --
> 2.16.3.windows.1
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#57273): https://edk2.groups.io/g/devel/message/57273
Mute This Topic: https://groups.io/mt/72729664/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux

2020-04-02 Thread Bob Feng 
This change is good to me.

Reviewed-by: Bob C Feng 

-Original Message-
From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Michael 
Kubacki
Sent: Friday, April 3, 2020 1:05 AM
To: devel@edk2.groups.io
Cc: Feng, Bob C ; Gao, Liming 
Subject: [edk2-devel] [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use 
multiple threads on Linux

From: Sean Brogan 

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2640

Azure Pipelines agents have 2 threads. This commit has been shown to reduce the 
build time in half on those agents.

Cc: Bob C Feng 
Cc: Liming Gao 
Signed-off-by: Michael Kubacki 
---
 BaseTools/Edk2ToolsBuild.py | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py index 
057d2e9e0633..1ea8187de693 100644
--- a/BaseTools/Edk2ToolsBuild.py
+++ b/BaseTools/Edk2ToolsBuild.py
@@ -11,6 +11,7 @@ import os
 import sys
 import logging
 import argparse
+import multiprocessing
 from edk2toolext import edk2_logging
 from edk2toolext.environment import self_describing_environment  from 
edk2toolext.base_abstract_invocable import BaseAbstractInvocable @@ -141,7 
+142,8 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
 return ret
 
 elif self.tool_chain_tag.lower().startswith("gcc"):
-ret = RunCmd("make", "-C .", 
workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+cpu_count = self.GetCpuThreads()
+ret = RunCmd("make", f"-C .  -j {cpu_count}", 
+ workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
 if ret != 0:
 raise Exception("Failed to build.")
 
@@ -154,6 +156,18 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
 logging.critical("Tool Chain not supported")
 return -1
 
+def GetCpuThreads(self) -> int:
+''' Function to return number of cpus. If error return 1'''
+cpus = 1
+try:
+cpus = multiprocessing.cpu_count()
+except:
+# from the internet there are cases where cpu_count is not 
implemented.
+# will handle error by just doing single proc build
+pass
+return cpus
+
+
 
 def main():
 Edk2ToolsBuild().Invoke()
--
2.16.3.windows.1





-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#56922): https://edk2.groups.io/g/devel/message/56922
Mute This Topic: https://groups.io/mt/72729664/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH v1 1/1] BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux

2020-04-02 Thread Michael Kubacki 
From: Sean Brogan 

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2640

Azure Pipelines agents have 2 threads. This commit has been shown to
reduce the build time in half on those agents.

Cc: Bob C Feng 
Cc: Liming Gao 
Signed-off-by: Michael Kubacki 
---
 BaseTools/Edk2ToolsBuild.py | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py
index 057d2e9e0633..1ea8187de693 100644
--- a/BaseTools/Edk2ToolsBuild.py
+++ b/BaseTools/Edk2ToolsBuild.py
@@ -11,6 +11,7 @@ import os
 import sys
 import logging
 import argparse
+import multiprocessing
 from edk2toolext import edk2_logging
 from edk2toolext.environment import self_describing_environment
 from edk2toolext.base_abstract_invocable import BaseAbstractInvocable
@@ -141,7 +142,8 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
 return ret
 
 elif self.tool_chain_tag.lower().startswith("gcc"):
-ret = RunCmd("make", "-C .", 
workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+cpu_count = self.GetCpuThreads()
+ret = RunCmd("make", f"-C .  -j {cpu_count}", 
workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
 if ret != 0:
 raise Exception("Failed to build.")
 
@@ -154,6 +156,18 @@ class Edk2ToolsBuild(BaseAbstractInvocable):
 logging.critical("Tool Chain not supported")
 return -1
 
+def GetCpuThreads(self) -> int:
+''' Function to return number of cpus. If error return 1'''
+cpus = 1
+try:
+cpus = multiprocessing.cpu_count()
+except:
+# from the internet there are cases where cpu_count is not 
implemented.
+# will handle error by just doing single proc build
+pass
+return cpus
+
+
 
 def main():
 Edk2ToolsBuild().Invoke()
-- 
2.16.3.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#56907): https://edk2.groups.io/g/devel/message/56907
Mute This Topic: https://groups.io/mt/72729664/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-