Gabe Black has submitted this change. (
https://gem5-review.googlesource.com/c/public/gem5/+/32124 )
Change subject: scons,fastmodel: Limit how many instances of simgen can run
at once.
......................................................................
scons,fastmodel: Limit how many instances of simgen can run at once.
Each instance of simgen uses a license. If there are only so many to
go around, running many instances at once could exhaust the pool of
licenses and break the build.
The number of licenses may be less than the number of regular build
steps we want to do in parallel, but may be greater than zero. To
limit them to at most n in parallel where n might be less than j
and/or more than 1, we create a group of license slots, assign simgen
invocations to a slot, and then use scons's side effect mechanism to
ensure no two invocations in the same slot run at the same time.
This may be a suboptimal packing if the commands take significantly
different amounts of time to run since the slots are preallocated and
not demand allocated, but the difference shouldn't normally matter in
practice, and scons doesn't provide a better mechanism for partially
serializing certain build steps.
Change-Id: Ifae58b48ae1b989c1915444bf7564f352f042305
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/32124
Reviewed-by: Bobby R. Bruce <bbr...@ucdavis.edu>
Maintainer: Gabe Black <gabebl...@google.com>
Tested-by: kokoro <noreply+kok...@google.com>
---
M src/arch/arm/fastmodel/SConscript
M src/arch/arm/fastmodel/SConsopts
2 files changed, 15 insertions(+), 0 deletions(-)
Approvals:
Bobby R. Bruce: Looks good to me, approved
Gabe Black: Looks good to me, approved
kokoro: Regressions pass
diff --git a/src/arch/arm/fastmodel/SConscript
b/src/arch/arm/fastmodel/SConscript
index c9b08b1..3980373 100644
--- a/src/arch/arm/fastmodel/SConscript
+++ b/src/arch/arm/fastmodel/SConscript
@@ -36,6 +36,7 @@
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
from __future__ import print_function
+from itertools import cycle
Import('*')
@@ -242,6 +243,10 @@
t[0] = t[1]
+license_count = int(env['ARMLMD_LICENSE_COUNT'])
+arm_licenses = list((Value(i) for i in range(license_count)))
+license_cycle = cycle(arm_licenses)
+
class ArmFastModelComponent(object):
def __init__(self, project_file, *extra_deps):
project_file = File(project_file)
@@ -312,6 +317,14 @@
sources = [project_file]
sources.extend(extra_deps)
env.Command(lib_nodes + self.headers, sources, simgen_action)
+ # Distribute simgen actions among ARM license slots. All actions
which
+ # have a given license as a "side effect" will be serialized
relative
+ # to each other, meaning the number of licenses being used
concurrently
+ # will never be larger than the number of license nodes.
+ #
+ # This allocation is fixed and may not be as optimal as a dynamic
one,
+ # but the difference is probably not significant.
+ env.SideEffect(next(license_cycle), lib_nodes[0])
def prepare_env(self, env):
env.Append(LIBPATH=self.libpaths)
diff --git a/src/arch/arm/fastmodel/SConsopts
b/src/arch/arm/fastmodel/SConsopts
index 35e3895..74165fe 100644
--- a/src/arch/arm/fastmodel/SConsopts
+++ b/src/arch/arm/fastmodel/SConsopts
@@ -41,6 +41,8 @@
os.environ.get('MAXCORE_HOME', '')),
('ARMLMD_LICENSE_FILE', 'ARM license file location',
os.environ.get('ARMLMD_LICENSE_FILE', '')),
+ ('ARMLMD_LICENSE_COUNT',
+ 'The maximum number of ARM licenses to use concurrently', 1),
('SIMGEN', 'simgen executable', os.environ.get('SIMGEN',
default_simgen)),
)
--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/32124
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings
Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: Ifae58b48ae1b989c1915444bf7564f352f042305
Gerrit-Change-Number: 32124
Gerrit-PatchSet: 4
Gerrit-Owner: Gabe Black <gabebl...@google.com>
Gerrit-Reviewer: Andreas Sandberg <andreas.sandb...@arm.com>
Gerrit-Reviewer: Bobby R. Bruce <bbr...@ucdavis.edu>
Gerrit-Reviewer: Gabe Black <gabebl...@google.com>
Gerrit-Reviewer: Giacomo Travaglini <giacomo.travagl...@arm.com>
Gerrit-Reviewer: kokoro <noreply+kok...@google.com>
Gerrit-MessageType: merged
_______________________________________________
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
