[gem5-dev] Change in gem5/gem5[develop]: base: Notify stats output of first/last dump
Jason Lowe-Power has uploaded this change for review. (
https://gem5-review.googlesource.com/c/public/gem5/+/35576 )
Change subject: base: Notify stats output of first/last dump
..
base: Notify stats output of first/last dump
To have multiple stats dumps in a single json file, we need to use a
json array. However, to have a json array, we need to know when the
first stat dump happens and the last at the json output. This changeset
adds this new interface to the stats output object, plumbs these calls
into the python code and implements them in the json output.
Change-Id: I482fd330af23b4a5f690739a81853eb5e2518839
Signed-off-by: Jason Lowe-Power
---
M src/base/stats/json.cc
M src/base/stats/json.hh
M src/base/stats/output.hh
M src/python/m5/simulate.py
M src/python/m5/stats/__init__.py
M src/python/pybind11/stats.cc
6 files changed, 51 insertions(+), 1 deletion(-)
diff --git a/src/base/stats/json.cc b/src/base/stats/json.cc
index 0dc5472..c97df81 100644
--- a/src/base/stats/json.cc
+++ b/src/base/stats/json.cc
@@ -36,8 +36,24 @@
{
void
+Json::firstDump()
+{
+ccprintf(*stream, "[");
+}
+
+void
+Json::lastDump()
+{
+ccprintf(*stream, "]");
+}
+
+void
Json::begin()
{
+printComma();
+
+_first.push(true);
+
ccprintf(*stream, "{");
}
diff --git a/src/base/stats/json.hh b/src/base/stats/json.hh
index 9723211..7cd5cd4 100644
--- a/src/base/stats/json.hh
+++ b/src/base/stats/json.hh
@@ -72,6 +72,8 @@
void begin() override;
void end() override;
+void firstDump() override;
+void lastDump() override;
};
Output *initJson(const std::string &filename);
diff --git a/src/base/stats/output.hh b/src/base/stats/output.hh
index 6ff4a5d..7ad413c 100644
--- a/src/base/stats/output.hh
+++ b/src/base/stats/output.hh
@@ -59,6 +59,20 @@
{
virtual ~Output() {}
+/**
+ * This function is used to signal to the output that this is the first
+ * time that "dump" has been called. Used in formats where the first
dump
+ * must output a special character (e.g., beginning of an array in
json)
+ */
+virtual void firstDump() { }
+
+/**
+ * This function is used to signal to the output that this is the last
+ * time that "dump" will be called. Used in formats where the last dump
+ * must output a special character (e.g., end of an array in json)
+ */
+virtual void lastDump() { }
+
virtual void begin() = 0;
virtual void end() = 0;
virtual bool valid() const = 0;
diff --git a/src/python/m5/simulate.py b/src/python/m5/simulate.py
index 080d725..e39735e 100644
--- a/src/python/m5/simulate.py
+++ b/src/python/m5/simulate.py
@@ -164,6 +164,7 @@
# Python exit handlers happen in reverse order.
# We want to dump stats last.
+atexit.register(stats.finalDump)
atexit.register(stats.dump)
# register our C++ exit callback function with Python
diff --git a/src/python/m5/stats/__init__.py
b/src/python/m5/stats/__init__.py
index 20867da..9a721e4 100644
--- a/src/python/m5/stats/__init__.py
+++ b/src/python/m5/stats/__init__.py
@@ -382,8 +382,14 @@
global global_dump_roots
all_roots.extend(global_dump_roots)
-now = m5.curTick()
global lastDump
+
+if lastDump == 0:
+first_dump = True
+else:
+first_dump = False
+
+now = m5.curTick()
assert lastDump <= now
new_dump = lastDump != now
lastDump = now
@@ -404,10 +410,19 @@
for output in outputList:
if output.valid():
+if first_dump:
+output.firstDump()
output.begin()
_dump_to_visitor(output, roots=all_roots)
output.end()
+def finalDump():
+"""Calls lastDump() on all outputs to finalize the output files
+"""
+for output in outputList:
+if output.valid():
+output.lastDump()
+
def reset():
'''Reset all statistics to the base state'''
diff --git a/src/python/pybind11/stats.cc b/src/python/pybind11/stats.cc
index 8ac46c2..9e0ff4b 100644
--- a/src/python/pybind11/stats.cc
+++ b/src/python/pybind11/stats.cc
@@ -117,6 +117,8 @@
;
py::class_(m, "Output")
+.def("firstDump", &Stats::Output::firstDump)
+.def("lastDump", &Stats::Output::lastDump)
.def("begin", &Stats::Output::begin)
.def("end", &Stats::Output::end)
.def("valid", &Stats::Output::valid)
--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/35576
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings
Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I482fd330af23b4a5f690739a81853eb5e2518839
Gerrit-Change-Number: 35576
Gerrit-PatchSet: 1
Gerrit-Owner: Jason Lowe-Power
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscr
[gem5-dev] Change in gem5/gem5[develop]: base: Add JSON output for stats
Jason Lowe-Power has uploaded this change for review. (
https://gem5-review.googlesource.com/c/public/gem5/+/35575 )
Change subject: base: Add JSON output for stats
..
base: Add JSON output for stats
This patch adds the initial code to allow JSON file output for stats.
No options, yet. All "new style" stats are correctly nested under their
parents, but the "old style" are just keys containing their entire
name. Only scalar stats are currently supported.
Change-Id: If6f98fbad7f61dd6f87ac9c409e54e2724807de6
Signed-off-by: Jason Lowe-Power
---
M src/base/SConscript
A src/base/stats/json.cc
A src/base/stats/json.hh
M src/python/m5/stats/__init__.py
M src/python/pybind11/stats.cc
5 files changed, 262 insertions(+), 0 deletions(-)
diff --git a/src/base/SConscript b/src/base/SConscript
index e04d84a..c464ab2 100644
--- a/src/base/SConscript
+++ b/src/base/SConscript
@@ -76,6 +76,7 @@
GTest('uncontended_mutex.test', 'uncontended_mutex.test.cc')
Source('stats/group.cc')
+Source('stats/json.cc')
Source('stats/text.cc')
if env['USE_HDF5']:
if main['GCC']:
diff --git a/src/base/stats/json.cc b/src/base/stats/json.cc
new file mode 100644
index 000..0dc5472
--- /dev/null
+++ b/src/base/stats/json.cc
@@ -0,0 +1,161 @@
+/* Copyright (c) 2020 The Regents of The University of California
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met: redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer;
+ * redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution;
+ * neither the name of the copyright holders nor the names of its
+ * contributors may be used to endorse or promote products derived from
+ * this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+
+#include "base/stats/json.hh"
+
+#include "base/logging.hh"
+#include "base/stats/info.hh"
+#include "base/str.hh"
+
+namespace Stats
+{
+
+void
+Json::begin()
+{
+ccprintf(*stream, "{");
+}
+
+void
+Json::end()
+{
+ccprintf(*stream, "}");
+}
+
+void
+Json::printComma()
+{
+// Note: json doesn't allow trailing commas. So, we are going to put a
+// comma *before* every entry except the first
+if (!_first.top()) {
+ccprintf(*stream, ",");
+} else {
+// If this is the first element in the section, replace the
top "true"
+// with false.
+_first.pop();
+_first.push(false);
+}
+}
+
+void
+Json::beginGroup(const char *name)
+{
+printComma();
+
+_first.push(true);
+
+ccprintf(*stream, "\"%s\":{", name);
+}
+
+void
+Json::endGroup()
+{
+_first.pop();
+ccprintf(*stream, "}");
+}
+
+void
+Json::visit(const ScalarInfo &info)
+{
+if (noOutput(info))
+return;
+
+printComma();
+
+ccprintf(*stream, "\"%s\":%s", info.name,
+ ValueToString(info.result(), info.precision));
+}
+
+void
+Json::visit(const VectorInfo &info)
+{
+if (noOutput(info))
+return;
+
+warn_once("json output does not support vectors");
+}
+
+void
+Json::visit(const DistInfo &info)
+{
+if (noOutput(info))
+return;
+
+warn_once("json output does not support distributions");
+}
+
+void
+Json::visit(const VectorDistInfo &info)
+{
+if (noOutput(info))
+return;
+
+warn_once("json output does not support vector distributions");
+}
+
+void
+Json::visit(const Vector2dInfo &info)
+{
+if (noOutput(info))
+return;
+
+warn_once("json output does not support 2D vectors");
+}
+
+void
+Json::visit(const FormulaInfo &info)
+{
+if (noOutput(info))
+return;
+
+warn_once("json output does not support formulas");
+}
+
+void
+Json::visit(const SparseHistInfo &info)
+{
+if (noOutput(info))
+return;
+
+warn_once("json output does not support sparse histograms");
+}
+
+Output *initJson(const std:
[gem5-dev] Change in gem5/gem5[develop]: cpu: Add recursion for DTB entry generation inside BaseCPU
Pierre Ayoub has uploaded this change for review. ( https://gem5-review.googlesource.com/c/public/gem5/+/35556 ) Change subject: cpu: Add recursion for DTB entry generation inside BaseCPU .. cpu: Add recursion for DTB entry generation inside BaseCPU Change-Id: Ice93b67ee44a1228120f8a63ad5b9d952f813c70 --- M src/cpu/BaseCPU.py 1 file changed, 6 insertions(+), 0 deletions(-) diff --git a/src/cpu/BaseCPU.py b/src/cpu/BaseCPU.py index ad91f3a..138bf25 100644 --- a/src/cpu/BaseCPU.py +++ b/src/cpu/BaseCPU.py @@ -299,6 +299,12 @@ node.appendPhandle(phandle_key) cpus_node.append(node) +# Generate nodes from the BaseCPU children (hence under the root node, +# and don't add them as subnode). Please note: this is mainly needed +# for the ISA class, to generate the PMU entry in the DTB. +for child_node in self.recurseDeviceTree(state): +yield child_node + yield cpus_node def __init__(self, **kwargs): -- To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/35556 To unsubscribe, or for help writing mail filters, visit https://gem5-review.googlesource.com/settings Gerrit-Project: public/gem5 Gerrit-Branch: develop Gerrit-Change-Id: Ice93b67ee44a1228120f8a63ad5b9d952f813c70 Gerrit-Change-Number: 35556 Gerrit-PatchSet: 1 Gerrit-Owner: Pierre Ayoub Gerrit-MessageType: newchange ___ gem5-dev mailing list -- gem5-dev@gem5.org To unsubscribe send an email to gem5-dev-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-dev] Change in gem5/gem5[develop]: arch-arm: Add recursion for DTB entry generation inside ArmISA
Pierre Ayoub has uploaded this change for review. ( https://gem5-review.googlesource.com/c/public/gem5/+/3 ) Change subject: arch-arm: Add recursion for DTB entry generation inside ArmISA .. arch-arm: Add recursion for DTB entry generation inside ArmISA In order to generate the ArmPMU's DTB entry, we have to enable recursion from the ArmISA. This commit follows this mailing list entry: https://www.mail-archive.com/gem5-users@gem5.org/msg18401.html Change-Id: I73012755f0f8c8d4d17278793cf16cb1e8b011df --- M src/arch/arm/ArmISA.py 1 file changed, 5 insertions(+), 0 deletions(-) diff --git a/src/arch/arm/ArmISA.py b/src/arch/arm/ArmISA.py index ebad774..e23ef2b 100644 --- a/src/arch/arm/ArmISA.py +++ b/src/arch/arm/ArmISA.py @@ -36,6 +36,7 @@ from m5.params import * from m5.proxy import * +from m5.SimObject import SimObject from m5.objects.ArmPMU import ArmPMU from m5.objects.ArmSystem import SveVectorLength from m5.objects.BaseISA import BaseISA @@ -55,6 +56,10 @@ decoderFlavor = Param.DecoderFlavor( 'Generic', "Decoder flavor specification") +# Recurse into subnodes to generate DTB entries. This is mainly needed to +# generate the PMU entry. +generateDeviceTree = SimObject.recurseDeviceTree + # If no MIDR value is provided, 0x0 is treated by gem5 as follows: # When 'highest_el_is_64' (AArch64 support) is: # True -> Cortex-A57 TRM r0p0 MIDR is used -- To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/3 To unsubscribe, or for help writing mail filters, visit https://gem5-review.googlesource.com/settings Gerrit-Project: public/gem5 Gerrit-Branch: develop Gerrit-Change-Id: I73012755f0f8c8d4d17278793cf16cb1e8b011df Gerrit-Change-Number: 3 Gerrit-PatchSet: 1 Gerrit-Owner: Pierre Ayoub Gerrit-MessageType: newchange ___ gem5-dev mailing list -- gem5-dev@gem5.org To unsubscribe send an email to gem5-dev-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
