[gem5-dev] Re: DPRINTF_UNCONDITIONAL deprecation

[Gabe Black via gem5-dev]

On Wed, Jul 21, 2021 at 8:26 AM Giacomo Travaglini <
giacomo.travagl...@arm.com> wrote:

> Thanks Gabe,
>
> The performance benefit is difficult to measure as the macro is not
> currently used; it could provide negligible or no improvement at all
> depending on the compiler and on where it could be used.
> So the question is not whether it makes gem5 faster now, it is more about
> giving our users the chance to write *potentially* faster code.
>


I'm not convinced by this because of two things. First, this is saying that
there may be a hypothetical example in the future where we *could* see a
benefit from this. I don't think just because something *could* be
hypothetically useful in the future that that's a good enough reason to
keep it. Second, while the performance benefit from this code, even if it
wasn't guarding logging, would likely be very small in an absolute sense if
anything, it is *also* immediately preceding code which will write to a log
file, to the console, etc, (since the presumption is this code *will*
output if we get to it), and that is a very high latency sort of operation.
That means that we're really talking about a very small or possibly
non-existent performance improvement, and stacking it on top of a very slow
operation that will completely overwhelm that difference. It's like taking
the radio out of a dump truck to save weight, maybe useful in a race car,
not useful in these circumstances.



>
> As you were saying the debug::Flag check could be removed by a later
> change.
> I'd argue though that a user should be able to catch the unexpected
> prints. In some cases, those prints would be even useful, signalling some
> debug only logic
> has slipped within the non-tracing code. If we take human error into
> consideration I am more worried about the dangers of silently executing
> debug only
> code even when no debug flag is specified.
>
> To provide a concrete example from the before mentioned patch [1], it
> means executing the printQueue method without printing anything.
> This won't happen if we use a DPRINTF_UNCONDITIONAL
>


Well, the problem is that it *will* happen, it's just that the user will
have some evidence that it's happening. It would still be up to them to
notice and do something about it. Also if they *do* need to call that
function (not necessarily the one in your example) for some reason which
doesn't have to do with debugging, then they have to either put up with the
log output, or make changes to the underlying code that they then need to
carry forward as a patch, or try to upstream, or... If it just does what
it's supposed to do, they don't have to mess with it and it will just work.

It's very reasonable to try to enforce expectations for a particular piece
of code like you're saying though. I think in that case you could use an
assert or other automatic mechanism which would more clearly and
definitively blow up when something was wrong, rather than something that
would just behave strangely or suspiciously, with the hope that someone
investigates it to find the problem.



>
> Please let me know your thoughts,
>
> Kind Regards
>
> Giacomo
>
> [1]:
> https://gem5-review.googlesource.com/c/public/gem5/+/47199/7/src/mem/cache/prefetch/queued.cc
>
>
> > -Original Message-
> > From: Gabe Black via gem5-dev 
> > Sent: 15 July 2021 23:30
> > To: gem5 Developer List 
> > Cc: Gabe Black 
> > Subject: [gem5-dev] Re: DPRINTF_UNCONDITIONAL deprecation
> >
> > Meant to reply to the list.
> >
> > On Thu, Jul 15, 2021 at 3:29 PM Gabe Black  >  > wrote:
> >
> >
> >   Hi Giacomo. The DPRINTF_UNCONDITIONAL macro was used in one
> > place, and that I converted that place to a normal DPRINTF. Please see
> here
> > for explanation for that change, and justification for this type of
> change in
> > general:
> >
> >   commit 07074f40e5a7e6000262a5d37f0be836f6ac92a9
> >   Author: Gabe Black  >  >
> >   Date:   Thu Apr 29 20:09:32 2021 -0700
> >
> >   There are several problems with DPRINTF_UNCONDITIONAL. First,
> > while macros don't take any run time overhead because they just turn into
> > c++, and of course you could have just typed that c++ to begin with,
> they do
> > have overhead because they make the code more complex and fragile, and
> > obfuscate what's actually going on. This isn't cost incurred (directly)
> by the
> > computer, but by the people who work on the code. This may eventually
> > translate into cost to the computer, because when code is harder to work
> > with, it's less likely to be completely correct or optimized.
> >
> >   Second, the compiler is much better at figuring out and keeping
> track
> > of what code is reachable each time you build. Code paths change, new
> > callers are added, etc, and what may have been already guarded by a debug
> > variable may not any more. Also, the compiler can already merge together
> or
> > drop checks it knows are unnecessary, 

[gem5-dev] Re: kokoro failures due to bug in gem5-art?

[Gabe Black via gem5-dev]

Ok thanks, Bobby. Please let me know if you find anything, especially if it
looks like it's a bug in kokoro itself somehow.

Gabe

On Wed, Jul 21, 2021 at 3:52 PM Bobby Bruce  wrote:

> There's definitely something funny going on with the gem5art tests there
> but I believe that error is happening without triggering a non-zero exit
> code. The gem5art test script is set to `set -e`, which means the script
> should exit immediately after a failure, yet it doesn't. The testing also
> continues onto the other tests. I'll look into this.
>
> In the example you linked, the issue appears to be because it has reached
> the 6 hour timeout. We could increase the timeout to fix this, but I'd like
> to know why our build/test times have increased enough to push us over the
> 6 hour line.  I'll see if I can figure it out as well.
>
> --
> Dr. Bobby R. Bruce
> Room 3050,
> Kemper Hall, UC Davis
> Davis,
> CA, 95616
>
> web: https://www.bobbybruce.net
>
>
> On Wed, Jul 21, 2021 at 2:51 PM Gabe Black via gem5-dev 
> wrote:
>
>> I've seen many kokoro failures lately, including this one which seems to
>> be from a problem in gem5-art? Any idea what's going on?
>>
>>
>> https://source.cloud.google.com/results/invocations/caae5aad-91a6-4c6e-9fbe-20962f9c5519/targets/gem5%2Fgcp_ubuntu%2Fpresubmit/log
>> ___
>> gem5-dev mailing list -- gem5-dev@gem5.org
>> To unsubscribe send an email to gem5-dev-le...@gem5.org
>> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
>
>
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Re: kokoro failures due to bug in gem5-art?

[Bobby Bruce via gem5-dev]

There's definitely something funny going on with the gem5art tests there
but I believe that error is happening without triggering a non-zero exit
code. The gem5art test script is set to `set -e`, which means the script
should exit immediately after a failure, yet it doesn't. The testing also
continues onto the other tests. I'll look into this.

In the example you linked, the issue appears to be because it has reached
the 6 hour timeout. We could increase the timeout to fix this, but I'd like
to know why our build/test times have increased enough to push us over the
6 hour line.  I'll see if I can figure it out as well.

--
Dr. Bobby R. Bruce
Room 3050,
Kemper Hall, UC Davis
Davis,
CA, 95616

web: https://www.bobbybruce.net


On Wed, Jul 21, 2021 at 2:51 PM Gabe Black via gem5-dev 
wrote:

> I've seen many kokoro failures lately, including this one which seems to
> be from a problem in gem5-art? Any idea what's going on?
>
>
> https://source.cloud.google.com/results/invocations/caae5aad-91a6-4c6e-9fbe-20962f9c5519/targets/gem5%2Fgcp_ubuntu%2Fpresubmit/log
> ___
> gem5-dev mailing list -- gem5-dev@gem5.org
> To unsubscribe send an email to gem5-dev-le...@gem5.org
> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] kokoro failures due to bug in gem5-art?

[Gabe Black via gem5-dev]

I've seen many kokoro failures lately, including this one which seems to be
from a problem in gem5-art? Any idea what's going on?

https://source.cloud.google.com/results/invocations/caae5aad-91a6-4c6e-9fbe-20962f9c5519/targets/gem5%2Fgcp_ubuntu%2Fpresubmit/log
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Re: DPRINTF_UNCONDITIONAL deprecation

[Giacomo Travaglini via gem5-dev]

Thanks Gabe,

The performance benefit is difficult to measure as the macro is not currently 
used; it could provide negligible or no improvement at all depending on the 
compiler and on where it could be used.
So the question is not whether it makes gem5 faster now, it is more about 
giving our users the chance to write *potentially* faster code.

As you were saying the debug::Flag check could be removed by a later change.
I'd argue though that a user should be able to catch the unexpected prints. In 
some cases, those prints would be even useful, signalling some debug only logic
has slipped within the non-tracing code. If we take human error into 
consideration I am more worried about the dangers of silently executing debug 
only
code even when no debug flag is specified.

To provide a concrete example from the before mentioned patch [1], it means 
executing the printQueue method without printing anything.
This won't happen if we use a DPRINTF_UNCONDITIONAL

Please let me know your thoughts,

Kind Regards

Giacomo

[1]: 
https://gem5-review.googlesource.com/c/public/gem5/+/47199/7/src/mem/cache/prefetch/queued.cc


> -Original Message-
> From: Gabe Black via gem5-dev 
> Sent: 15 July 2021 23:30
> To: gem5 Developer List 
> Cc: Gabe Black 
> Subject: [gem5-dev] Re: DPRINTF_UNCONDITIONAL deprecation
>
> Meant to reply to the list.
>
> On Thu, Jul 15, 2021 at 3:29 PM Gabe Black   > wrote:
>
>
>   Hi Giacomo. The DPRINTF_UNCONDITIONAL macro was used in one
> place, and that I converted that place to a normal DPRINTF. Please see here
> for explanation for that change, and justification for this type of change in
> general:
>
>   commit 07074f40e5a7e6000262a5d37f0be836f6ac92a9
>   Author: Gabe Black   >
>   Date:   Thu Apr 29 20:09:32 2021 -0700
>
>   There are several problems with DPRINTF_UNCONDITIONAL. First,
> while macros don't take any run time overhead because they just turn into
> c++, and of course you could have just typed that c++ to begin with, they do
> have overhead because they make the code more complex and fragile, and
> obfuscate what's actually going on. This isn't cost incurred (directly) by the
> computer, but by the people who work on the code. This may eventually
> translate into cost to the computer, because when code is harder to work
> with, it's less likely to be completely correct or optimized.
>
>   Second, the compiler is much better at figuring out and keeping track
> of what code is reachable each time you build. Code paths change, new
> callers are added, etc, and what may have been already guarded by a debug
> variable may not any more. Also, the compiler can already merge together or
> drop checks it knows are unnecessary, so I think we may frequently be
> dropping something the compiler is already going to drop for us. I think
> DPRINTF_UNCONDITIONAL may be a bit of premature optimization without
> measurement to say that it's really necessary or even useful.
>
>   I acknowledge that the lack of performance benefit is also an
> assumption on my part though, and please let me know if you can measure a
> difference in the scenarios you're thinking of. I still think the complexity 
> and
> fragility argument apply, but if the difference is substantial it might be 
> still be
> worth it.
>
>   Gabe
>
>
>
>   On Thu, Jul 15, 2021 at 6:08 AM Giacomo Travaglini
> mailto:giacomo.travagl...@arm.com> >
> wrote:
>
>
>   Hi Gabe,
>
>   The DPRINTF_UNCONDITIONAL macro has been deprecated
> in develop [1] and  the deprecation is going to be official in v21.1.
>   As far as I understood, the reason why it has been
> deprecated is because it was not used in gem5.
>
>   I am proposing to remove the deprecation. IMO that macro is
> useful when the Debug flag has already been checked outside of the
>   Debug print. We recently merged a patch [2] that would have
> benefitted from the use of DPRINTF_UNCONDITIONAL. I am quite confident
>   I can find some other parts of gem5 where the
> DPRINTF_UNCONDITIONAL would have been a better printing choice.
>   I believe speed should still be a prime concern even if we are
> tracing, to make the debugging process faster and smoother for our users,
> especially
>   As it comes with no cost (the definition of a macro)
>
>   Let me know what you think about this, and if you don't have
> any objection I will push a revert of the patch you posted, and I will convert
>   some DPRINT into a DPRINTF_UNCONDITIONAL
>
>   Kind Regards
>
>   Giacomo
>
>   [1]: https://gem5-
> review.googlesource.com/c/public/gem5/+/44987
>   [2]: https://gem5-
> review.googlesource.com/c/public/gem5/+/47199/7/src/mem/cache/prefet
> ch/queued.cc#134
>   IMPORTANT NOTICE: The contents of this email and any
> 

[gem5-dev] Change in gem5/gem5[develop]: cpu: Remove unnecessary includes of arch/locked_mem.hh.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48383 )



Change subject: cpu: Remove unnecessary includes of arch/locked_mem.hh.
..

cpu: Remove unnecessary includes of arch/locked_mem.hh.

Change-Id: I85769ea286e5ecc77ac7d7db1b09cb4b87129cd4
---
M src/cpu/minor/execute.cc
M src/cpu/o3/lsq_unit.hh
2 files changed, 0 insertions(+), 2 deletions(-)



diff --git a/src/cpu/minor/execute.cc b/src/cpu/minor/execute.cc
index 81850cb..3234de5 100644
--- a/src/cpu/minor/execute.cc
+++ b/src/cpu/minor/execute.cc
@@ -39,7 +39,6 @@

 #include 

-#include "arch/locked_mem.hh"
 #include "cpu/minor/cpu.hh"
 #include "cpu/minor/exec_context.hh"
 #include "cpu/minor/fetch1.hh"
diff --git a/src/cpu/o3/lsq_unit.hh b/src/cpu/o3/lsq_unit.hh
index 88e9849..686ca16 100644
--- a/src/cpu/o3/lsq_unit.hh
+++ b/src/cpu/o3/lsq_unit.hh
@@ -50,7 +50,6 @@

 #include "arch/generic/debugfaults.hh"
 #include "arch/generic/vec_reg.hh"
-#include "arch/locked_mem.hh"
 #include "base/circular_queue.hh"
 #include "config/the_isa.hh"
 #include "cpu/base.hh"

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48383
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I85769ea286e5ecc77ac7d7db1b09cb4b87129cd4
Gerrit-Change-Number: 48383
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: arch-x86: De-conditionalize segmentation microops.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has submitted this change. (  
https://gem5-review.googlesource.com/c/public/gem5/+/45305 )


Change subject: arch-x86: De-conditionalize segmentation microops.
..

arch-x86: De-conditionalize segmentation microops.

These were never used with conditions, so the condition check just added
overhead. Also, the not-taken path through the instruction didn't
actually set the destination to something, meaning that it would set it
to something arbitrary and not actually leave it unmodified.

Change-Id: I33fef088979b14ad74adf22b26419a1cacf386dd
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/45305
Tested-by: kokoro 
Reviewed-by: Gabe Black 
Maintainer: Gabe Black 
---
M src/arch/x86/isa/microops/regop.isa
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Gabe Black: Looks good to me, approved; Looks good to me, approved
  kokoro: Regressions pass



diff --git a/src/arch/x86/isa/microops/regop.isa  
b/src/arch/x86/isa/microops/regop.isa

index 0cca5b5..10bf8f7 100644
--- a/src/arch/x86/isa/microops/regop.isa
+++ b/src/arch/x86/isa/microops/regop.isa
@@ -1400,7 +1400,7 @@
 '''

 # Microops for manipulating segmentation registers
-class SegOp(CondRegOp):
+class SegOp(RegOp):
 abstract = True
 operand_types = (SegDestOp, FoldedSrc1Op)
 def __init__(self, dest, src1, flags=None,  
dataSize="env.dataSize"):




2 is the latest approved patch-set.
No files were changed between the latest approved patch-set and the  
submitted one.

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/45305
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I33fef088979b14ad74adf22b26419a1cacf386dd
Gerrit-Change-Number: 45305
Gerrit-PatchSet: 4
Gerrit-Owner: Gabe Black 
Gerrit-Reviewer: Gabe Black 
Gerrit-Reviewer: Hoa Nguyen 
Gerrit-Reviewer: Jason Lowe-Power 
Gerrit-Reviewer: Pouya Fotouhi 
Gerrit-Reviewer: kokoro 
Gerrit-MessageType: merged
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: sim: Use a range based for loop in EmbeddedPython::initAll.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48366 )



Change subject: sim: Use a range based for loop in EmbeddedPython::initAll.
..

sim: Use a range based for loop in EmbeddedPython::initAll.

Change-Id: I380bed880735a411c6069079b4ae38a9d9080744
---
M src/sim/init.cc
1 file changed, 3 insertions(+), 5 deletions(-)



diff --git a/src/sim/init.cc b/src/sim/init.cc
index 238bf54..b6ba35e 100644
--- a/src/sim/init.cc
+++ b/src/sim/init.cc
@@ -142,12 +142,10 @@

 // Load the rest of the embedded python files into the embedded
 // python importer
-std::list::iterator i = getList().begin();
-std::list::iterator end = getList().end();
-for (; i != end; ++i)
-if (!(*i)->addModule())
+for (auto *embedded: getList()) {
+if (!embedded->addModule())
 return 1;
-
+}
 return 0;
 }


--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48366
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I380bed880735a411c6069079b4ae38a9d9080744
Gerrit-Change-Number: 48366
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: python: Minor cleanups in the marshal program source.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48382 )



Change subject: python: Minor cleanups in the marshal program source.
..

python: Minor cleanups in the marshal program source.

Fix some minor style issues, use a "raw" string constant to make the
marshal script more readable, get rid of a redundant \n in the help
text, and make argv const.

Change-Id: I1dc3181a67b50286e3a0b833bb7251b7efd01978
---
M src/python/marshal.cc
1 file changed, 15 insertions(+), 9 deletions(-)



diff --git a/src/python/marshal.cc b/src/python/marshal.cc
index 394b783..7e05c3a 100644
--- a/src/python/marshal.cc
+++ b/src/python/marshal.cc
@@ -42,23 +42,29 @@
 namespace py = pybind11;
 using namespace pybind11::literals;

+constexpr auto MarshalScript = R"(
+import marshal
+
+with open(source, 'r') as f:
+src = f.read()
+
+compiled = compile(src, source, 'exec')
+marshalled = marshal.dumps(compiled)
+)";
+
 int
-main(int argc, char **argv) {
-py::scoped_interpreter guard{};
+main(int argc, const char **argv)
+{
+py::scoped_interpreter guard;

 if (argc != 2) {
-std::cerr << "Usage: marshal PYSOURCE\n" << std::endl;
+std::cerr << "Usage: marshal PYSOURCE" << std::endl;
 exit(1);
 }

 auto locals = py::dict("source"_a=argv[1]);

-py::exec(
-"import marshal\n"
-"with open(source, 'r') as f: src = f.read()\n"
-"compiled = compile(src, source, 'exec')\n"
-"marshalled = marshal.dumps(compiled)\n",
-py::globals(), locals);
+py::exec(MarshalScript, py::globals(), locals);

 auto marshalled = locals["marshalled"].cast();
 std::cout << marshalled;

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48382
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I1dc3181a67b50286e3a0b833bb7251b7efd01978
Gerrit-Change-Number: 48382
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons: Clean up the definition of m5.defines a little bit.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48381 )



Change subject: scons: Clean up the definition of m5.defines a little bit.
..

scons: Clean up the definition of m5.defines a little bit.

Use the new helper functions to go to/from a Value(), and tidy things up
slightly.

Change-Id: I9a31004b5a610bb8e94848d1fb88606dda6fc3c2
---
M src/SConscript
1 file changed, 2 insertions(+), 5 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index 66a5c56..ffefac2 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -728,15 +728,12 @@
 # Generate Python file containing a dict specifying the current
 # buildEnv flags.
 def makeDefinesPyFile(target, source, env):
-build_env = source[0].get_contents().decode('utf-8')
-
 code = code_formatter()
-code("buildEnv = dict($build_env)")
+code("buildEnv = $0", FromValue(source[0]))
 code.write(target[0].abspath)

-defines_info = Value(build_env)
 # Generate a file with all of the compile options in it
-env.Command('python/m5/defines.py', defines_info,
+env.Command('python/m5/defines.py', ToValue(dict(build_env)),
 MakeAction(makeDefinesPyFile, Transform("DEFINES", 0)))
 PySource('m5', 'python/m5/defines.py')


--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48381
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I9a31004b5a610bb8e94848d1fb88606dda6fc3c2
Gerrit-Change-Number: 48381
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons: Eliminate flag_* entries from m5.defines.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48379 )



Change subject: scons: Eliminate flag_* entries from m5.defines.
..

scons: Eliminate flag_* entries from m5.defines.

These are not used anywhere, and are very old.

Change-Id: If37a8fe2e0c3374fba1930353e502746f333d86d
---
M src/SConscript
M src/python/pybind11/core.cc
2 files changed, 0 insertions(+), 24 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index d508cbf..5339f20 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -733,18 +733,11 @@
 code = code_formatter()
 code("""
 import _m5.core
-import m5.util

 buildEnv = dict($build_env)

 compileDate = _m5.core.compileDate
 gem5Version = _m5.core.gem5Version
-_globals = globals()
-for key,val in _m5.core.__dict__.items():
-if key.startswith('flag_'):
-flag = key[5:]
-_globals[flag] = val
-del _globals
 """)
 code.write(target[0].abspath)

diff --git a/src/python/pybind11/core.cc b/src/python/pybind11/core.cc
index 0e9c311..1bca941 100644
--- a/src/python/pybind11/core.cc
+++ b/src/python/pybind11/core.cc
@@ -86,18 +86,6 @@
 extern const char *compileDate;
 extern const char *gem5Version;

-#ifdef DEBUG
-const bool flag_DEBUG = true;
-#else
-const bool flag_DEBUG = false;
-#endif
-#ifdef NDEBUG
-const bool flag_NDEBUG = true;
-#else
-const bool flag_NDEBUG = false;
-#endif
-const bool flag_TRACING_ON = TRACING_ON;
-
 static void
 init_drain(py::module_ _native)
 {
@@ -300,11 +288,6 @@
 m_core.attr("compileDate") = py::cast(compileDate);
 m_core.attr("gem5Version") = py::cast(gem5Version);

-m_core.attr("flag_DEBUG") = py::cast(flag_DEBUG);
-m_core.attr("flag_DEBUG") = py::cast(flag_DEBUG);
-m_core.attr("flag_NDEBUG") = py::cast(flag_NDEBUG);
-m_core.attr("flag_TRACING_ON") = py::cast(flag_TRACING_ON);
-
 m_core.attr("MaxTick") = py::cast(MaxTick);

 /*

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48379
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: If37a8fe2e0c3374fba1930353e502746f333d86d
Gerrit-Change-Number: 48379
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons: Pull the "Blob" builder out of src/SConscript.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48378 )



Change subject: scons: Pull the "Blob" builder out of src/SConscript.
..

scons: Pull the "Blob" builder out of src/SConscript.

Change-Id: Ib52c7b51d52aeccdcd2ca05cb0a71267268d969d
---
M SConstruct
M site_scons/gem5_scons/builders/__init__.py
A site_scons/gem5_scons/builders/blob.py
M src/SConscript
4 files changed, 109 insertions(+), 61 deletions(-)



diff --git a/SConstruct b/SConstruct
index 1dd0a8d..3233e56 100755
--- a/SConstruct
+++ b/SConstruct
@@ -130,6 +130,7 @@
 from gem5_scons import TempFileSpawn, EnvDefaults, MakeAction,  
MakeActionTool

 import gem5_scons
 from gem5_scons.builders import ConfigFile, AddLocalRPATH, SwitchingHeaders
+from gem5_scons.builders import Blob
 from gem5_scons.sources import TagImpliesTool
 from gem5_scons.util import compareVersions, readCommand

@@ -150,7 +151,7 @@

 main = Environment(tools=[
 'default', 'git', TempFileSpawn, EnvDefaults, MakeActionTool,
-ConfigFile, AddLocalRPATH, SwitchingHeaders, TagImpliesTool
+ConfigFile, AddLocalRPATH, SwitchingHeaders, TagImpliesTool, Blob
 ])

 main.Tool(SCons.Tool.FindTool(['gcc', 'clang'], main))
diff --git a/site_scons/gem5_scons/builders/__init__.py  
b/site_scons/gem5_scons/builders/__init__.py

index 1130e42..d8e8d46 100755
--- a/site_scons/gem5_scons/builders/__init__.py
+++ b/site_scons/gem5_scons/builders/__init__.py
@@ -25,5 +25,6 @@


 from .add_local_rpath import AddLocalRPATH
+from .blob import Blob
 from .config_file import ConfigFile
 from .switching_headers import SwitchingHeaders
diff --git a/site_scons/gem5_scons/builders/blob.py  
b/site_scons/gem5_scons/builders/blob.py

new file mode 100644
index 000..ac3ed67
--- /dev/null
+++ b/site_scons/gem5_scons/builders/blob.py
@@ -0,0 +1,106 @@
+# -*- mode:python -*-
+
+# Copyright (c) 2018, 2020 ARM Limited
+#
+# The license below extends only to copyright in the software and shall
+# not be construed as granting a license to any other intellectual
+# property including but not limited to intellectual property relating
+# to a hardware implementation of the functionality of the software
+# licensed hereunder.  You may use the software subject to the license
+# terms below provided that you ensure that this notice is replicated
+# unmodified and in its entirety in all distributions of the software,
+# modified or unmodified, in source code or in binary form.
+#
+# Copyright (c) 2004-2005 The Regents of The University of Michigan
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import os.path
+
+from gem5_scons import Transform, MakeAction
+from gem5_scons.util import bytesToCppArray
+
+from m5.util import code_formatter
+
+def build_blob(target, source, env):
+'''
+Embed an arbitrary blob into the gem5 executable,
+and make it accessible to C++ as a byte array.
+'''
+
+with open(str(source[0]), 'rb') as f:
+data = f.read()
+symbol = str(source[1])
+cc, hh = target
+
+hh_code = code_formatter()
+hh_code('''\
+#include 
+#include 
+
+namespace gem5
+{
+namespace Blobs
+{
+
+extern const std::size_t ${symbol}_len;
+extern const std::uint8_t ${symbol}[];
+
+} // namespace Blobs
+} // namespace gem5
+''')
+hh_code.write(str(hh))
+
+include_path = os.path.relpath(hh.abspath, env['BUILDDIR'])
+
+cc_code = code_formatter()
+cc_code('''\
+#include 

[gem5-dev] Change in gem5/gem5[develop]: scons: Move the source related helper classes out of src/SConscript.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48376 )



Change subject: scons: Move the source related helper classes out of  
src/SConscript.

..

scons: Move the source related helper classes out of src/SConscript.

By having them in gem5_scons.sources, they can be used by mechanisms
outside of src/SConscript, like separated out builders.

Change-Id: Ic3769723c8413e7db48aef536572ad3f2f948658
---
M SConstruct
A site_scons/gem5_scons/sources.py
M src/SConscript
3 files changed, 244 insertions(+), 197 deletions(-)



diff --git a/SConstruct b/SConstruct
index 7deb6e4..1dd0a8d 100755
--- a/SConstruct
+++ b/SConstruct
@@ -130,6 +130,7 @@
 from gem5_scons import TempFileSpawn, EnvDefaults, MakeAction,  
MakeActionTool

 import gem5_scons
 from gem5_scons.builders import ConfigFile, AddLocalRPATH, SwitchingHeaders
+from gem5_scons.sources import TagImpliesTool
 from gem5_scons.util import compareVersions, readCommand

 # Disable warnings when targets can be built with multiple environments but
@@ -149,7 +150,7 @@

 main = Environment(tools=[
 'default', 'git', TempFileSpawn, EnvDefaults, MakeActionTool,
-ConfigFile, AddLocalRPATH, SwitchingHeaders
+ConfigFile, AddLocalRPATH, SwitchingHeaders, TagImpliesTool
 ])

 main.Tool(SCons.Tool.FindTool(['gcc', 'clang'], main))
diff --git a/site_scons/gem5_scons/sources.py  
b/site_scons/gem5_scons/sources.py

new file mode 100644
index 000..6f44041
--- /dev/null
+++ b/site_scons/gem5_scons/sources.py
@@ -0,0 +1,239 @@
+# -*- mode:python -*-
+
+# Copyright (c) 2018, 2020 ARM Limited
+#
+# The license below extends only to copyright in the software and shall
+# not be construed as granting a license to any other intellectual
+# property including but not limited to intellectual property relating
+# to a hardware implementation of the functionality of the software
+# licensed hereunder.  You may use the software subject to the license
+# terms below provided that you ensure that this notice is replicated
+# unmodified and in its entirety in all distributions of the software,
+# modified or unmodified, in source code or in binary form.
+#
+# Copyright (c) 2004-2005 The Regents of The University of Michigan
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import functools
+
+import SCons.Script
+
+
+# Code for adding source files of various types
+#
+# When specifying a source file of some type, a set of tags can be
+# specified for that file.
+
+def tag_implies(env, tag, tag_list):
+'''
+Associates a tag X to a list of tags which are implied by X.
+
+For example, assume:
+- Each file .cc is tagged with the tag "Tag ".
+- B.cc refers to symbols from A.cc
+- C.cc refers to symbols from B.cc
+- D.cc refers to symbols from A.cc and C.cc
+
+Then:
+- "Tag A" is implied by "Tag B"
+- "Tag B" is implied by "Tag C"
+- "Tag A" is transitively implied by "Tag C" (from "Tag B")
+- "Tag A" and "Tag C" are implied by "Tag D"
+- "Tag B" is transitively implied by "Tag D" (from "Tag C")
+- "Tag A" is transitively implied by "Tag D" (from transitive "Tag B")
+
+All of these implications are simply declared as:
+env.TagImplies("Tag B", "Tag A")
+env.TagImplies("Tag C", "Tag B")
+env.TagImplies("Tag D", ["Tag A", "Tag C"])
+
+So that any use of 

[gem5-dev] Change in gem5/gem5[develop]: scons,python: Stop importing some values in m5.defines.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48380 )



Change subject: scons,python: Stop importing some values in m5.defines.
..

scons,python: Stop importing some values in m5.defines.

The compileDate and gem5Version fields are used in only one place,
gem5's python main function. These fields are the remaining difference
between the "fake" defines.py provided by the SimObject importer, and
the real one composed later. It makes sense to exclude them in the
"fake" version since those values come from c++, but it would feel like
an arbitrary and unexpected difference to people trying to use it.

Change-Id: Ie344765bf7c8063197da24f5b55f762379deff94
---
M src/SConscript
M src/python/m5/main.py
2 files changed, 4 insertions(+), 10 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index 5339f20..66a5c56 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -731,14 +731,7 @@
 build_env = source[0].get_contents().decode('utf-8')

 code = code_formatter()
-code("""
-import _m5.core
-
-buildEnv = dict($build_env)
-
-compileDate = _m5.core.compileDate
-gem5Version = _m5.core.gem5Version
-""")
+code("buildEnv = dict($build_env)")
 code.write(target[0].abspath)

 defines_info = Value(build_env)
diff --git a/src/python/m5/main.py b/src/python/m5/main.py
index 9342ad0..4fcdf70 100644
--- a/src/python/m5/main.py
+++ b/src/python/m5/main.py
@@ -213,6 +213,7 @@

 def main(*args):
 import m5
+import _m5.core

 from . import core
 from . import debug
@@ -335,8 +336,8 @@
 print(brief_copyright)
 print()

-print("gem5 version %s" % defines.gem5Version)
-print("gem5 compiled %s" % defines.compileDate)
+print("gem5 version %s" % _m5.core.gem5Version)
+print("gem5 compiled %s" % _m5.core.compileDate)

 print("gem5 started %s" %
   datetime.datetime.now().strftime("%b %e %Y %X"))

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48380
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: Ie344765bf7c8063197da24f5b55f762379deff94
Gerrit-Change-Number: 48380
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons: Move the bytesToCppArray helper to gem5_scons.util.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48377 )



Change subject: scons: Move the bytesToCppArray helper to gem5_scons.util.
..

scons: Move the bytesToCppArray helper to gem5_scons.util.

Change-Id: Ib8789dd33ebbfb8e10446de5d1079654a2200d2d
---
M site_scons/gem5_scons/util.py
M src/SConscript
2 files changed, 15 insertions(+), 14 deletions(-)



diff --git a/site_scons/gem5_scons/util.py b/site_scons/gem5_scons/util.py
index b62cc01..517e584 100644
--- a/site_scons/gem5_scons/util.py
+++ b/site_scons/gem5_scons/util.py
@@ -38,6 +38,7 @@
 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

+import array
 import itertools
 import re
 import sys
@@ -109,3 +110,16 @@
 if n1 > n2: return  1

 return 0
+
+def bytesToCppArray(code, symbol, data):
+'''
+Output an array of bytes to a code formatter as a c++ array  
declaration.

+'''
+code('const std::uint8_t ${symbol}[] = {')
+code.indent()
+step = 16
+for i in range(0, len(data), step):
+x = array.array('B', data[i:i+step])
+code(''.join('%d,' % d for d in x))
+code.dedent()
+code('};')
diff --git a/src/SConscript b/src/SConscript
index ae8bdbe..e7c5067 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -37,7 +37,6 @@
 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-import array
 import bisect
 import distutils.spawn
 import importlib
@@ -53,6 +52,7 @@

 from gem5_scons import Transform, warning, error, ToValue, FromValue
 from gem5_scons.sources import *
+from gem5_scons.util import bytesToCppArray

 Export(SourceFilter.factories)

@@ -68,19 +68,6 @@

 from m5.util import code_formatter

-def bytesToCppArray(code, symbol, data):
-'''
-Output an array of bytes to a code formatter as a c++ array  
declaration.

-'''
-code('const std::uint8_t ${symbol}[] = {')
-code.indent()
-step = 16
-for i in range(0, len(data), step):
-x = array.array('B', data[i:i+step])
-code(''.join('%d,' % d for d in x))
-code.dedent()
-code('};')
-
 def build_blob(target, source, env):
 '''
 Embed an arbitrary blob into the gem5 executable,

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48377
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: Ib8789dd33ebbfb8e10446de5d1079654a2200d2d
Gerrit-Change-Number: 48377
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons: Build the source filter factories dict in SourceFilter.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48375 )



Change subject: scons: Build the source filter factories dict in  
SourceFilter.

..

scons: Build the source filter factories dict in SourceFilter.

This is a little cleaner since it avoids an additional global variable.

Change-Id: I19d9a0afd12fdfeeda0b524bd71943d155ed5d7d
---
M src/SConscript
A src/proto/hello.proto
2 files changed, 18 insertions(+), 9 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index e914bbd..6da1bf0 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -148,6 +148,7 @@
 return tags

 class SourceFilter(object):
+factories = {}
 def __init__(self, predicate):
 self.predicate = predicate

@@ -159,10 +160,6 @@
 return SourceFilter(lambda env, tags: self.predicate(env, tags) and
   other.predicate(env, tags))

-def with_tags_that(predicate):
-'''Return a list of sources with tags that satisfy a predicate.'''
-return SourceFilter(predicate)
-
 def with_any_tags(*tags):
 '''Return a list of sources with any of the supplied tags.'''
 return SourceFilter(lambda env, stags: \
@@ -185,16 +182,15 @@
 '''Return a list of sources without the supplied tag.'''
 return without_tags(*[tag])

-source_filter_factories = {
-'with_tags_that': with_tags_that,
+SourceFilter.factories.update({
 'with_any_tags': with_any_tags,
 'with_all_tags': with_all_tags,
 'with_tag': with_tag,
 'without_tags': without_tags,
 'without_tag': without_tag,
-}
+})

-Export(source_filter_factories)
+Export(SourceFilter.factories)

 class SourceList(list):
 def apply_filter(self, env, f):
@@ -203,7 +199,7 @@
 return SourceList(filter(match, self))

 def __getattr__(self, name):
-func = source_filter_factories.get(name, None)
+func = SourceFilter.factories.get(name, None)
 if not func:
 raise AttributeError

diff --git a/src/proto/hello.proto b/src/proto/hello.proto
new file mode 100644
index 000..f2490f3
--- /dev/null
+++ b/src/proto/hello.proto
@@ -0,0 +1,13 @@
+syntax = "proto3";
+
+service HelloService {
+  rpc SayHello (HelloRequest) returns (HelloResponse);
+}
+
+message HelloRequest {
+  string greeting = 1;
+}
+
+message HelloResponse {
+  string reply = 1;
+}

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48375
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I19d9a0afd12fdfeeda0b524bd71943d155ed5d7d
Gerrit-Change-Number: 48375
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons: Turn gem5_scons.builders from a module into a package.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48374 )



Change subject: scons: Turn gem5_scons.builders from a module into a  
package.

..

scons: Turn gem5_scons.builders from a module into a package.

This will make it easier to organize when there are more builders in the
future.

Change-Id: I3db2f2c87b39ed0d427a2e505c7c9152e47ee58b
---
A site_scons/gem5_scons/builders/__init__.py
A site_scons/gem5_scons/builders/add_local_rpath.py
R site_scons/gem5_scons/builders/config_file.py
A site_scons/gem5_scons/builders/switching_headers.py
4 files changed, 174 insertions(+), 69 deletions(-)



diff --git a/site_scons/gem5_scons/builders/__init__.py  
b/site_scons/gem5_scons/builders/__init__.py

new file mode 100755
index 000..1130e42
--- /dev/null
+++ b/site_scons/gem5_scons/builders/__init__.py
@@ -0,0 +1,29 @@
+# Copyright Google, Inc.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+from .add_local_rpath import AddLocalRPATH
+from .config_file import ConfigFile
+from .switching_headers import SwitchingHeaders
diff --git a/site_scons/gem5_scons/builders/add_local_rpath.py  
b/site_scons/gem5_scons/builders/add_local_rpath.py

new file mode 100755
index 000..ce26575
--- /dev/null
+++ b/site_scons/gem5_scons/builders/add_local_rpath.py
@@ -0,0 +1,68 @@
+# Copyright (c) 2013, 2015-2020 ARM Limited
+# All rights reserved.
+#
+# The license below extends only to copyright in the software and shall
+# not be construed as granting a license to any other intellectual
+# property including but not limited to intellectual property relating
+# to a hardware implementation of the functionality of the software
+# licensed hereunder.  You may use the software subject to the license
+# terms below provided that you ensure that this notice is replicated
+# unmodified and in its entirety in all distributions of the software,
+# modified or unmodified, in source code or in binary form.
+#
+# Copyright (c) 2011 Advanced Micro Devices, Inc.
+# Copyright (c) 2009 The Hewlett-Packard Development Company
+# Copyright (c) 2004-2005 The Regents of The University of Michigan
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, 

[gem5-dev] Change in gem5/gem5[develop]: scons: Tidy up the definition of SourceFile slightly.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48373 )



Change subject: scons: Tidy up the definition of SourceFile slightly.
..

scons: Tidy up the definition of SourceFile slightly.

Use {} notation for creating a set, and rely on the fact that applying
File() to something that already is does nothing.

Change-Id: I2ec99e4a4859df9a0a88bcc38e93233841124de6
---
M src/SConscript
1 file changed, 3 insertions(+), 5 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index 51e053c..e914bbd 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -229,23 +229,21 @@
 if tags is None:
 tags='gem5 lib'
 if isinstance(tags, str):
-tags = set([tags])
+tags = { tags }
 if not isinstance(tags, set):
 tags = set(tags)
 self.tags = tags

 if add_tags:
 if isinstance(add_tags, str):
-add_tags = set([add_tags])
+add_tags = { add_tags }
 if not isinstance(add_tags, set):
 add_tags = set(add_tags)
 self.tags |= add_tags

 self.append = append

-tnode = source
-if not isinstance(source, SCons.Node.FS.File):
-tnode = File(source)
+tnode = File(source)

 self.tnode = tnode
 self.filename = str(self.tnode)

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48373
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I2ec99e4a4859df9a0a88bcc38e93233841124de6
Gerrit-Change-Number: 48373
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons: Define the rules for building debug flag hdrs in place.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48372 )



Change subject: scons: Define the rules for building debug flag hdrs in  
place.

..

scons: Define the rules for building debug flag hdrs in place.

Define the rules for building debug flag header files in place, instead
of looping over them all after they've been accumulated.

Change-Id: I02113a21529958c3f971b5462ea340d70d1b18d7
---
M src/SConscript
1 file changed, 55 insertions(+), 56 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index 96a2997..51e053c 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -53,7 +53,7 @@

 import SCons

-from gem5_scons import Transform, warning, error
+from gem5_scons import Transform, warning, error, ToValue, FromValue

 # This file defines how to build a particular configuration of gem5
 # based on variable settings in the 'env' build environment.
@@ -669,13 +669,59 @@
 #
 # Debug Flags
 #
+
+def makeDebugFlagHH(target, source, env):
+assert len(target) == 1 and len(source) == 1
+
+name, components, desc, fmt = FromValue(source[0])
+
+code = code_formatter()
+
+typename = "CompoundFlag" if components else "SimpleFlag"
+component_flag_decls = \
+''.join('extern SimpleFlag& %s;\n' % flag for flag in  
components)

+
+# file header boilerplate
+code('''\
+#ifndef __DEBUG_${name}_HH__
+#define __DEBUG_${name}_HH__
+
+#include "base/compiler.hh" // For namespace deprecation
+
+namespace gem5
+{
+
+GEM5_DEPRECATED_NAMESPACE(Debug, debug);
+namespace debug
+{
+
+class ${typename};
+extern ${typename}& ${name};
+${component_flag_decls}
+
+} // namespace debug
+} // namespace gem5
+
+#endif // __DEBUG_${name}_HH__
+''')
+
+code.write(str(target[0]))
+
 debug_flags = {}
 def DebugFlag(name, desc=None, fmt=False):
 if name == "All":
 raise AttributeError('The "All" flag name is reserved')
 if name in debug_flags:
-raise AttributeError('Flag {} already specified'.format(name))
-debug_flags[name] = (name, (), desc, fmt)
+raise AttributeError("Flag {} already specified".format(name))
+
+flag = (name, (), desc, fmt)
+debug_flags[name] = flag
+
+env.Append(SIMPLE_DEBUG_FLAGS=[name])
+
+hh_file = Dir(env['BUILDDIR']).Dir('debug').File('%s.hh' % name)
+env.Command(hh_file, ToValue(flag),
+MakeAction(makeDebugFlagHH, Transform("TRACING", 0)))

 def CompoundFlag(name, flags, desc=None):
 if name == "All":
@@ -683,8 +729,12 @@
 if name in debug_flags:
 raise AttributeError('Flag {} already specified'.format(name))

-compound = tuple(flags)
-debug_flags[name] = (name, compound, desc, False)
+flag = (name, flags, desc, False)
+debug_flags[name] = flag
+
+env.Command(Dir(env['BUILDDIR']).Dir('debug').File('%s.hh' % name),
+ToValue(flag),
+MakeAction(makeDebugFlagHH, Transform("TRACING", 0)))

 def DebugFormatFlag(name, desc=None):
 DebugFlag(name, desc, True)
@@ -1224,58 +1274,7 @@

 code.write(str(target[0]))

-def makeDebugFlagHH(target, source, env):
-assert(len(target) == 1 and len(source) == 1)
-
-val = eval(source[0].get_contents())
-name, compound, desc, fmt = val
-
-code = code_formatter()
-
-# file header boilerplate
-code('''\
-#ifndef __DEBUG_${name}_HH__
-#define __DEBUG_${name}_HH__
-
-#include "base/compiler.hh" // For namespace deprecation
-
-namespace gem5
-{
-
-GEM5_DEPRECATED_NAMESPACE(Debug, debug);
-namespace debug
-{
-''')
-
-if compound:
-code('class CompoundFlag;')
-code('class SimpleFlag;')
-
-if compound:
-code('extern CompoundFlag& $name;')
-for flag in compound:
-code('extern SimpleFlag& $flag;')
-else:
-code('extern SimpleFlag& $name;')
-
-code('''
-} // namespace debug
-} // namespace gem5
-
-#endif // __DEBUG_${name}_HH__
-''')
-
-code.write(str(target[0]))
-
 # Generate the files for the debug and debug-format flags
-for name,flag in sorted(debug_flags.items()):
-n, compound, desc, fmt = flag
-assert n == name
-
-hh_file = 'debug/%s.hh' % name
-env.Command(hh_file, Value(flag),
-MakeAction(makeDebugFlagHH, Transform("TRACING", 0)))
-
 env.Command('debug/flags.cc', Value(debug_flags),
 MakeAction(makeDebugFlagCC, Transform("TRACING", 0)))
 Source('debug/flags.cc', add_tags='gem5 trace')

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48372
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I02113a21529958c3f971b5462ea340d70d1b18d7
Gerrit-Change-Number: 48372
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- 

[gem5-dev] Change in gem5/gem5[develop]: scons: Add a pair of functions for working with Value nodes.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48371 )



Change subject: scons: Add a pair of functions for working with Value nodes.
..

scons: Add a pair of functions for working with Value nodes.

These nodes are really for working with text, and they do strange and
broken things otherwise. This change adds a pair of convenience
functions which uses pickle to serialize an object for the Value to
hold, and to unpack the Value later.

Change-Id: Id48822ce3de283b003d4cc7ef6b563a70e321ca6
---
M site_scons/gem5_scons/__init__.py
1 file changed, 9 insertions(+), 1 deletion(-)



diff --git a/site_scons/gem5_scons/__init__.py  
b/site_scons/gem5_scons/__init__.py

index 91764b0..bf7b85d 100644
--- a/site_scons/gem5_scons/__init__.py
+++ b/site_scons/gem5_scons/__init__.py
@@ -39,6 +39,7 @@
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

 import os
+import pickle
 import sys
 import tempfile
 import textwrap
@@ -46,6 +47,7 @@
 from gem5_scons.util import get_termcap
 from gem5_scons.configure import Configure
 from gem5_scons.defaults import EnvDefaults
+import SCons.Node.Python
 import SCons.Script

 termcap = get_termcap()
@@ -260,5 +262,11 @@
 env['SHCCCOMSTR']  = Transform("SHCC")
 env['SHCXXCOMSTR'] = Transform("SHCXX")

+def ToValue(obj):
+return SCons.Node.Python.Value(pickle.dumps(obj))
+
+def FromValue(node):
+return pickle.loads(node.read())
+
 __all__ = ['Configure', 'EnvDefaults', 'Transform', 'warning', 'error',
-   'MakeAction', 'MakeActionTool']
+   'MakeAction', 'MakeActionTool', 'ToValue', 'FromValue']

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48371
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: Id48822ce3de283b003d4cc7ef6b563a70e321ca6
Gerrit-Change-Number: 48371
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons,debug: Implement the "All" flag in C++ and not scons.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48370 )



Change subject: scons,debug: Implement the "All" flag in C++ and not scons.
..

scons,debug: Implement the "All" flag in C++ and not scons.

Create an AllFlagsFlag class which inherits from the CompoundFlag class.
This class is a singleton, and the SimpleFlags install themselves in it
instead of having SCons collect them.

The allFlagsVersion global variable was supposed to be for debugging
according to a comment, but was actually an important part of the "All"
flags inner workings. It was not exposed in the header, but was
redefined/pulled through in src/python/pybind11/debug.cc. The
AllFlagsFlag class now tracks that value, and it can be accessed without
reaching behind the curtain.

This also somewhat decentralizes the debug flag building process in
SCons. The debug/flags.cc still includes all flags at once which
centralizes them, but at least now the "All" flag won't also.

Change-Id: I8430e0fe9022846aade028fb46c80777169a2007
---
M src/SConscript
M src/base/debug.cc
M src/base/debug.hh
M src/python/pybind11/debug.cc
4 files changed, 61 insertions(+), 33 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index 99869a6..96a2997 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -671,13 +671,17 @@
 #
 debug_flags = {}
 def DebugFlag(name, desc=None, fmt=False):
+if name == "All":
+raise AttributeError('The "All" flag name is reserved')
 if name in debug_flags:
-raise AttributeError("Flag {} already specified".format(name))
+raise AttributeError('Flag {} already specified'.format(name))
 debug_flags[name] = (name, (), desc, fmt)

 def CompoundFlag(name, flags, desc=None):
+if name == "All":
+raise AttributeError('The "All" flag name is reserved')
 if name in debug_flags:
-raise AttributeError("Flag {} already specified".format(name))
+raise AttributeError('Flag {} already specified'.format(name))

 compound = tuple(flags)
 debug_flags[name] = (name, compound, desc, False)
@@ -685,19 +689,6 @@
 def DebugFormatFlag(name, desc=None):
 DebugFlag(name, desc, True)

-# Create a compound debug flag that encapsulates all flags: "All". This
-# flag should not be used within C++ code - it is a compound meta flag
-def _createAllDebugFlag():
-simple_flags = []
-for name,flag in sorted(debug_flags.items()):
-n, compound, desc, fmt = flag
-assert n == name
-if not compound and not fmt:
-simple_flags.append(n)
-
-CompoundFlag("All", simple_flags,
-"Controls all debug flags. It should not be used within C++ code.")
-
 Export('DebugFlag')
 Export('CompoundFlag')
 Export('DebugFormatFlag')
@@ -1277,7 +1268,6 @@
 code.write(str(target[0]))

 # Generate the files for the debug and debug-format flags
-_createAllDebugFlag()
 for name,flag in sorted(debug_flags.items()):
 n, compound, desc, fmt = flag
 assert n == name
diff --git a/src/base/debug.cc b/src/base/debug.cc
index 8e65e1f..aa4092a 100644
--- a/src/base/debug.cc
+++ b/src/base/debug.cc
@@ -71,13 +71,17 @@
 #endif
 }

-//
-// Flags for debugging purposes.  Primarily for trace.hh
-//
-int allFlagsVersion = 0;
+// Used to check the freshness of cached views of all flags.
 FlagsMap &
 allFlags()
 {
+// Ensure that the special "All" compound debug flag has been created,
+// and avoid infinite recursion.
+static bool done = false;
+if (!done) {
+done = true;
+AllFlagsFlag::instance();
+}
 static FlagsMap flags;
 return flags;
 }
@@ -88,8 +92,9 @@
 findFlag(const std::string )
 {
 FlagsMap::iterator i = allFlags().find(name);
-if (i == allFlags().end())
+if (i == allFlags().end()) {
 return NULL;
+}
 return i->second;
 }

@@ -101,8 +106,6 @@

 panic_if(!result.second, "Flag %s already defined!", name);

-++allFlagsVersion;
-
 sync();
 }

@@ -127,6 +130,14 @@
 i.second->sync();
 }

+SimpleFlag::SimpleFlag(const char *name, const char *desc, bool is_format)
+  : Flag(name, desc), _isFormat(is_format)
+{
+// Add non-format flags to the special "All" compound flag.
+if (!isFormat())
+AllFlagsFlag::instance().add(this);
+}
+
 void
 CompoundFlag::enable()
 {
@@ -141,6 +152,26 @@
 k->disable();
 }

+AllFlagsFlag::AllFlagsFlag() : CompoundFlag("All",
+"Controls all debug flags. It should not be used within C++  
code.", {})

+{}
+
+void
+AllFlagsFlag::add(SimpleFlag *flag)
+{
+++_version;
+_kids.push_back(flag);
+}
+
+int AllFlagsFlag::_version = 0;
+
+AllFlagsFlag &
+AllFlagsFlag::instance()
+{
+static AllFlagsFlag flag;
+return flag;
+}
+
 bool
 changeFlag(const char *s, bool value)
 {
diff --git a/src/base/debug.hh b/src/base/debug.hh
index d2dfee2..f6b03ae 100644
--- a/src/base/debug.hh
+++ 

[gem5-dev] Change in gem5/gem5[develop]: configs,python: Clean some cruft out of m5.objects.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48364 )



Change subject: configs,python: Clean some cruft out of m5.objects.
..

configs,python: Clean some cruft out of m5.objects.

SimObject is already available as m5.SimObject, and it doesn't make a
lot of sense to expose m5.internal.params, part of the internals of
gem5's python interface, as a peer to all the SimObject types.

Change-Id: I3030c1eb261877fd9648c9d3d73b7dbbd4c24345
---
M configs/common/ObjectList.py
M src/python/m5/objects/__init__.py
M src/python/m5/simulate.py
3 files changed, 6 insertions(+), 17 deletions(-)



diff --git a/configs/common/ObjectList.py b/configs/common/ObjectList.py
index aa3bda0..685dbc1 100644
--- a/configs/common/ObjectList.py
+++ b/configs/common/ObjectList.py
@@ -35,6 +35,7 @@
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

 import m5.objects
+import m5.internal.params
 import inspect
 import sys
 from textwrap import TextWrapper
diff --git a/src/python/m5/objects/__init__.py  
b/src/python/m5/objects/__init__.py

index e59f9a8..4bec74df 100644
--- a/src/python/m5/objects/__init__.py
+++ b/src/python/m5/objects/__init__.py
@@ -24,14 +24,6 @@
 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-from m5.internal import params
-from m5.SimObject import *
-
-try:
-modules = __spec__.loader_state
-except NameError:
-modules = { }
-
-for module in modules:
+for module in __spec__.loader_state:
 if module.startswith('m5.objects.'):
 exec("from %s import *" % module)
diff --git a/src/python/m5/simulate.py b/src/python/m5/simulate.py
index 4a05095..66e6a08 100644
--- a/src/python/m5/simulate.py
+++ b/src/python/m5/simulate.py
@@ -50,6 +50,7 @@
 from . import SimObject
 from . import ticks
 from . import objects
+from . import params
 from m5.util.dot_writer import do_dot, do_dvfs_dot
 from m5.util.dot_writer_ruby import do_ruby_dot

@@ -59,12 +60,6 @@
 # define a MaxTick parameter, unsigned 64 bit
 MaxTick = 2**64 - 1

-_memory_modes = {
-"atomic" : objects.params.atomic,
-"timing" : objects.params.timing,
-"atomic_noncaching" : objects.params.atomic_noncaching,
-}
-
 _drain_manager = _m5.drain.DrainManager.instance()

 # The final hook to generate .ini files.  Called from the user script
@@ -292,8 +287,9 @@
 raise RuntimeError(
 "Old CPU (%s) does not support CPU handover." % (old_cpu,))

+MemoryMode = params.allEnums["MemoryMode"]
 try:
-memory_mode = _memory_modes[memory_mode_name]
+memory_mode = MemoryMode(memory_mode_name).getValue()
 except KeyError:
 raise RuntimeError("Invalid memory mode (%s)" % memory_mode_name)

@@ -309,7 +305,7 @@
 # Flush the memory system if we are switching to a memory mode
 # that disables caches. This typically happens when switching to a
 # hardware virtualized CPU.
-if memory_mode == objects.params.atomic_noncaching:
+if memory_mode == MemoryMode("atomic_noncaching").getValue():
 memWriteback(system)
 memInvalidate(system)


--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48364
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I3030c1eb261877fd9648c9d3d73b7dbbd4c24345
Gerrit-Change-Number: 48364
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons: Generalize the Executable class to cover libraries too.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48369 )



Change subject: scons: Generalize the Executable class to cover libraries  
too.

..

scons: Generalize the Executable class to cover libraries too.

This way the shared and static gem5 libraries can be treated like other
top level build targets.

Change-Id: I04dd82f9be86df0a5cabd2e4934077c33235911c
---
M src/SConscript
1 file changed, 70 insertions(+), 81 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index 7c81f30..99869a6 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -527,23 +527,25 @@
 Source(env.ProtoBufCC(source=source)[0], tags=tags, add_tags=add_tags)


-exectuable_classes = []
-class ExecutableMeta(type):
-'''Meta class for Executables.'''
+
+date_source = File('base/date.cc')
+
+class TopLevelMeta(type):
+'''Meta class for top level build products, ie binaries and  
libraries.'''

 all = []

 def __init__(cls, name, bases, d):
-ExecutableMeta.all.append(cls)
-super(ExecutableMeta, cls).__init__(name, bases, d)
+TopLevelMeta.all.append(cls)
+super(TopLevelMeta, cls).__init__(name, bases, d)
 cls.all = []

-class Executable(object, metaclass=ExecutableMeta):
-'''Base class for creating an executable from sources.'''
+class TopLevelBase(object, metaclass=TopLevelMeta):
+'''Base class for linked build products.'''

 def __init__(self, target, *srcs_and_filts):
 '''Specify the target name and any sources. Sources that are
 not SourceFiles are evalued with Source().'''
-super(Executable, self).__init__()
+super(TopLevelBase, self).__init__()
 self.all.append(self)
 self.target = target

@@ -557,12 +559,12 @@
 src = Source(src, tags=[])
 srcs.append(src)

+for f in self.filters:
+srcs += Source.all.apply_filter(env, f)
+
 self.sources = srcs
 self.dir = Dir('.')

-def path(self, env):
-return self.dir.File(self.target + '.${EXE_SUFFIX}')
-
 def srcs_to_objs(self, env, sources):
 return list([ s.static(env) for s in sources ])

@@ -570,6 +572,37 @@
 def declare_all(cls, env):
 return list([ instance.declare(env) for instance in cls.all ])

+class StaticLib(TopLevelBase):
+'''Base class for creating a static library from sources.'''
+
+def declare(self, env):
+objs = self.srcs_to_objs(env, self.sources)
+
+date_obj = env.StaticObject(date_source)
+env.Depends(date_obj, objs)
+
+return env.StaticLibrary(self.target, [date_obj, objs])[0]
+
+class SharedLib(TopLevelBase):
+'''Base class for creating a shared library from sources.'''
+
+def srcs_to_objs(self, env, sources):
+return list([ s.shared(env) for s in sources ])
+
+def declare(self, env):
+objs = self.srcs_to_objs(env, self.sources)
+
+date_obj = env.SharedObject(date_source)
+env.Depends(date_obj, objs)
+
+return env.SharedLibrary(self.target, [date_obj, objs])[0]
+
+class Executable(TopLevelBase):
+'''Base class for creating an executable from sources.'''
+
+def path(self, env):
+return self.dir.File(self.target + '.${ENV_LABEL}')
+
 def declare(self, env, objs=None):
 if objs is None:
 objs = self.srcs_to_objs(env, self.sources)
@@ -578,7 +611,10 @@
 env['BIN_RPATH_PREFIX'] = os.path.relpath(
 env['BUILDDIR'], self.path(env).dir.abspath)

-executable = env.Program(self.path(env).abspath, objs)[0]
+date_obj = env.StaticObject(date_source)
+env.Depends(date_obj, objs)
+
+executable = env.Program(self.path(env).abspath, [date_obj,  
objs])[0]


 if sys.platform == 'sunos5':
 cmd = 'cp $SOURCE $TARGET; strip $TARGET'
@@ -593,10 +629,10 @@
 '''Create a unit test based on the google test framework.'''
 all = []
 def __init__(self, *srcs_and_filts, **kwargs):
+if not kwargs.pop('skip_lib', False):
+srcs_and_filts = srcs_and_filts + (with_tag('gtest lib'),)
 super(GTest, self).__init__(*srcs_and_filts)

-self.skip_lib = kwargs.pop('skip_lib', False)
-
 @classmethod
 def declare_all(cls, env):
 env = env.Clone()
@@ -604,20 +640,12 @@
 env['SHOBJSUFFIX'] = 't' + env['SHOBJSUFFIX']
 env.Append(LIBS=env['GTEST_LIBS'])
 env.Append(CPPFLAGS=env['GTEST_CPPFLAGS'])
-env['GTEST_LIB_SOURCES'] = Source.all.with_tag(env, 'gtest lib')
 env['GTEST_OUT_DIR'] = \
-Dir(env['BUILDDIR']).Dir('unittests.${EXE_SUFFIX}')
+Dir(env['BUILDDIR']).Dir('unittests.${ENV_LABEL}')
 return super(GTest, cls).declare_all(env)

 def declare(self, env):
-sources = list(self.sources)
-if not self.skip_lib:
-  

[gem5-dev] Change in gem5/gem5[develop]: scons: Get rid of special handling of the _m5 package.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48362 )



Change subject: scons: Get rid of special handling of the _m5 package.
..

scons: Get rid of special handling of the _m5 package.

This package is handled specially by the DictImporter used during the
build, and an assert shows that that code is never actually used. That
makes sense, since _m5 won't be added to gem5 using the PySource
mechanism.

Change-Id: I36a39f1ebb94a7620c8ba296e0fe856bd33285f9
---
M src/SConscript
1 file changed, 0 insertions(+), 3 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index 2284289..dbd28ea 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -787,9 +787,6 @@
 if fullname == 'm5.objects':
 return self

-if fullname.startswith('_m5'):
-return None
-
 source = self.modules.get(fullname, None)
 if source is not None and fullname.startswith('m5.objects'):
 return self

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48362
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I36a39f1ebb94a7620c8ba296e0fe856bd33285f9
Gerrit-Change-Number: 48362
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons: Get rid of a redundant import of code_formatter.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48361 )



Change subject: scons: Get rid of a redundant import of code_formatter.
..

scons: Get rid of a redundant import of code_formatter.

This module was already imported at the top of the SConscript.

Change-Id: I1357beda2adcc085c122df15397c769694a73fab
---
M src/SConscript
1 file changed, 0 insertions(+), 1 deletion(-)



diff --git a/src/SConscript b/src/SConscript
index 234e063..2284289 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -822,7 +822,6 @@

 import m5.SimObject
 import m5.params
-from m5.util import code_formatter

 m5.SimObject.clear()
 m5.params.clear()

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48361
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I1357beda2adcc085c122df15397c769694a73fab
Gerrit-Change-Number: 48361
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons,test: Don't -Wall or -Werror for googletest libraries.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48368 )



Change subject: scons,test: Don't -Wall or -Werror for googletest libraries.
..

scons,test: Don't -Wall or -Werror for googletest libraries.

These libraries come from elsewhere, and so there's no reason to worry
about warnings. We can't fix them even if they crop up. Also, set
CCFLAGS to avoid having a mixture of gem5 flags and googletest flags.

Change-Id: I19b07747a43cebb263ae1546c75631cff1f13132
---
M ext/googletest/SConscript
1 file changed, 2 insertions(+), 3 deletions(-)



diff --git a/ext/googletest/SConscript b/ext/googletest/SConscript
index e5241d6..5a4b691 100644
--- a/ext/googletest/SConscript
+++ b/ext/googletest/SConscript
@@ -43,9 +43,8 @@
 main.Append(CPPPATH=[gtest_include, gmock_include])
 main.Append(LIBPATH=[build])

-env = main.Clone(CPPFLAGS=['-g', '-Wall', '-Wextra', '-pthread',
-   '-Wno-undef', '-isystem', str(gtest_include),
-   '-isystem', str(gmock_include)])
+env = main.Clone(CCFLAGS=['-g', '-pthread', '-Wno-undef', '-isystem',
+  str(gtest_include), '-isystem',  
str(gmock_include)])

 env.Append(CPPPATH=[gtest_base, gmock_base])

 gtest_all = env.Object(gtest_src.File('gtest-all.cc'))

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48368
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I19b07747a43cebb263ae1546c75631cff1f13132
Gerrit-Change-Number: 48368
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: util: Sort the "updater tags" in cpt_upgrader.py --get-cc-file.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48360 )



Change subject: util: Sort the "updater tags" in cpt_upgrader.py  
--get-cc-file.

..

util: Sort the "updater tags" in cpt_upgrader.py --get-cc-file.

This ensures that the same tags will create the same file, and avoids
spurious rebuilds/relinks for null builds.

Change-Id: Ic8e37a24e2c60d74d8c921dde1c5e102d3a764e3
---
M util/cpt_upgrader.py
1 file changed, 1 insertion(+), 1 deletion(-)



diff --git a/util/cpt_upgrader.py b/util/cpt_upgrader.py
index e091af4..abbaba2 100755
--- a/util/cpt_upgrader.py
+++ b/util/cpt_upgrader.py
@@ -297,7 +297,7 @@
 print("{")
 print()
 print("std::set version_tags = {")
-for tag in Upgrader.tag_set:
+for tag in sorted(Upgrader.tag_set):
 print("  \"{}\",".format(tag))
 print("};")
 print()

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48360
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: Ic8e37a24e2c60d74d8c921dde1c5e102d3a764e3
Gerrit-Change-Number: 48360
Gerrit-PatchSet: 1
Gerrit-Owner: Gabe Black 
Gerrit-MessageType: newchange
___
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-dev] Change in gem5/gem5[develop]: scons,python,sim: Eliminate a redundant member of EmbeddedPython.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48365 )



Change subject: scons,python,sim: Eliminate a redundant member of  
EmbeddedPython.

..

scons,python,sim: Eliminate a redundant member of EmbeddedPython.

The filename member was just a less specific version of the abspath
member, and can be replaced by it to simplify things a little.

Change-Id: I61b312f2c356045e03462159e3232ac717954669
---
M src/SConscript
M src/python/importer.py
M src/sim/init.cc
M src/sim/init.hh
4 files changed, 12 insertions(+), 18 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index 34ba841..589a0e4 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -361,16 +361,14 @@
 assert ext == '.py'

 if package:
-path = package.split('.')
+modpath = package.split('.')
 else:
-path = []
+modpath = []

-modpath = path[:]
 if modname != '__init__':
 modpath += [ modname ]
 modpath = '.'.join(modpath)

-arcpath = path + [ basename ]
 abspath = self.snode.abspath
 if not os.path.exists(abspath):
 abspath = self.tnode.abspath
@@ -378,7 +376,6 @@
 self.package = package
 self.modname = modname
 self.modpath = modpath
-self.arcname = os.path.join(*arcpath)
 self.abspath = abspath
 self.cpp = File(self.filename + '.cc')

@@ -1263,7 +1260,6 @@
 bytesToCppArray(code, 'embedded_module_data', data)
 code('''
 EmbeddedPython embedded_module_info(
-${{c_str(pysource.arcname)}},
 ${{c_str(pysource.abspath)}},
 ${{c_str(pysource.modpath)}},
 embedded_module_data,
diff --git a/src/python/importer.py b/src/python/importer.py
index 4e5e907..76c0fa5 100644
--- a/src/python/importer.py
+++ b/src/python/importer.py
@@ -44,23 +44,23 @@
 override_var = os.environ.get('M5_OVERRIDE_PY_SOURCE', 'false')
 self.override = (override_var.lower() in ('true', 'yes'))

-def add_module(self, filename, abspath, modpath, code):
+def add_module(self, abspath, modpath, code):
 if modpath in self.modules:
 raise AttributeError("%s already found in importer" % modpath)

-self.modules[modpath] = (filename, abspath, code)
+self.modules[modpath] = (abspath, code)

 def find_spec(self, fullname, path, target=None):
 if fullname not in self.modules:
 return None

-srcfile, abspath, code = self.modules[fullname]
+abspath, code = self.modules[fullname]

 if self.override and os.path.exists(abspath):
 src = open(abspath, 'r').read()
 code = compile(src, abspath, 'exec')

-is_package = (os.path.basename(srcfile) == '__init__.py')
+is_package = (os.path.basename(abspath) == '__init__.py')
 spec = importlib.util.spec_from_loader(
 name=fullname, loader=ByteCodeLoader(code),
 is_package=is_package)
diff --git a/src/sim/init.cc b/src/sim/init.cc
index f9ab8b2..238bf54 100644
--- a/src/sim/init.cc
+++ b/src/sim/init.cc
@@ -75,10 +75,9 @@

 EmbeddedPython *EmbeddedPython::importer = NULL;
 PyObject *EmbeddedPython::importerModule = NULL;
-EmbeddedPython::EmbeddedPython(const char *filename, const char *abspath,
-const char *modpath, const unsigned char *code, int zlen, int len)
-: filename(filename), abspath(abspath), modpath(modpath), code(code),
-  zlen(zlen), len(len)
+EmbeddedPython::EmbeddedPython(const char *abspath, const char *modpath,
+const unsigned char *code, int zlen, int len)
+: abspath(abspath), modpath(modpath), code(code), zlen(zlen), len(len)
 {
 // if we've added the importer keep track of it because we need it
 // to bootstrap.
@@ -117,7 +116,7 @@
 {
 PyObject *code = getCode();
 PyObject *result = PyObject_CallMethod(importerModule,  
PyCC("add_module"),

-PyCC("sssO"), filename, abspath, modpath, code);
+PyCC("ssO"), abspath, modpath, code);
 if (!result) {
 PyErr_Print();
 return false;
diff --git a/src/sim/init.hh b/src/sim/init.hh
index a20d3b6..c2f4cf2 100644
--- a/src/sim/init.hh
+++ b/src/sim/init.hh
@@ -62,15 +62,14 @@
  */
 struct EmbeddedPython
 {
-const char *filename;
 const char *abspath;
 const char *modpath;
 const uint8_t *code;
 int zlen;
 int len;

-EmbeddedPython(const char *filename, const char *abspath,
-   const char *modpath, const uint8_t *code, int zlen, int  
len);

+EmbeddedPython(const char *abspath, const char *modpath,
+const uint8_t *code, int zlen, int len);

 PyObject *getCode() const;
 bool addModule() const;

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/48365
To unsubscribe, or for help writing mail filters, visit  

[gem5-dev] Change in gem5/gem5[develop]: scons: Pull some python related mechanisms out of USE_PYTHON guards.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48367 )



Change subject: scons: Pull some python related mechanisms out of  
USE_PYTHON guards.

..

scons: Pull some python related mechanisms out of USE_PYTHON guards.

We don't want to build certain files if USE_PYTHON is disabled, but we
can still tell scons how to.

Change-Id: I38c7c93f609cfcedc350f8270f0b239b69c4f101
---
M SConstruct
M src/SConscript
2 files changed, 69 insertions(+), 70 deletions(-)



diff --git a/SConstruct b/SConstruct
index 2b2e39c..7deb6e4 100755
--- a/SConstruct
+++ b/SConstruct
@@ -512,12 +512,6 @@
 if not py_version:
 error("Can't find a working Python installation")

-marshal_env = main.Clone()
-
-# Bare minimum environment that only includes python
-marshal_env.Append(CCFLAGS='$MARSHAL_CCFLAGS_EXTRA')
-marshal_env.Append(LINKFLAGS='$MARSHAL_LDFLAGS_EXTRA')
-
 # Found a working Python installation. Check if it meets minimum
 # requirements.
 ver_string = '.'.join(map(str, py_version))
@@ -528,6 +522,12 @@
 warning('Embedded python library too new. '
 'Python 3 expected, found %s.' % ver_string)

+marshal_env = main.Clone()
+
+# Bare minimum environment that only includes python
+marshal_env.Append(CCFLAGS='$MARSHAL_CCFLAGS_EXTRA')
+marshal_env.Append(LINKFLAGS='$MARSHAL_LDFLAGS_EXTRA')
+
 main['HAVE_PKG_CONFIG'] = main.Detect('pkg-config')

 with gem5_scons.Configure(main) as conf:
@@ -700,9 +700,7 @@
 env.Append(CCFLAGS='$CCFLAGS_EXTRA')
 env.Append(LINKFLAGS='$LDFLAGS_EXTRA')

-exports=['env']
-if main['USE_PYTHON']:
-exports.append('marshal_env')
+exports=['env', 'marshal_env']

 # The src/SConscript file sets up the build rules in 'env' according
 # to the configured variables.  It returns a list of environments,
diff --git a/src/SConscript b/src/SConscript
index 589a0e4..7c81f30 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -347,6 +347,64 @@
 class Source(SourceFile):
 pass

+# Build a small helper that marshals the Python code using the same version
+# of Python as gem5. This is in an unorthodox location to avoid building it
+# for every variant.
+py_marshal = marshal_env.Program('marshal', 'python/marshal.cc')[0]
+
+# Embed python files.  All .py files that have been indicated by a
+# PySource() call in a SConscript need to be embedded into the M5
+# library.  To do that, we compile the file to byte code, marshal the
+# byte code, compress it, and then generate a c++ file that
+# inserts the result into an array.
+def embedPyFile(target, source, env):
+def c_str(string):
+if string is None:
+return "0"
+return '"%s"' % string
+
+'''Action function to compile a .py into a code object, marshal it,
+compress it, and stick it into an asm file so the code appears as
+just bytes with a label in the data section. The action takes two
+sources:
+
+source[0]: Binary used to marshal Python sources
+source[1]: Python script to marshal
+'''
+
+import subprocess
+
+marshalled = subprocess.check_output(
+[source[0].abspath, str(source[1])], env=env['ENV'])
+
+compressed = zlib.compress(marshalled)
+data = compressed
+pysource = PySource.tnodes[source[1]]
+
+code = code_formatter()
+code('''\
+#include "sim/init.hh"
+
+namespace gem5
+{
+namespace
+{
+
+''')
+bytesToCppArray(code, 'embedded_module_data', data)
+code('''
+EmbeddedPython embedded_module_info(
+${{c_str(pysource.abspath)}},
+${{c_str(pysource.modpath)}},
+embedded_module_data,
+${{len(data)}},
+${{len(marshalled)}});
+
+} // anonymous namespace
+} // namespace gem5
+''')
+code.write(str(target[0]))
+
 class PySource(SourceFile):
 '''Add a python source file to the named package'''
 modules = {}
@@ -382,6 +440,9 @@
 PySource.modules[modpath] = self
 PySource.tnodes[self.tnode] = self

+marshal_env.Command(self.cpp, [ py_marshal, self.tnode ],
+MakeAction(embedPyFile, Transform("EMBED PY")))
+
 class SimObject(PySource):
 '''Add a SimObject python file as a python source object and add
 it to a list of sim object modules'''
@@ -1088,7 +1149,7 @@
 MakeAction(createSimObjectWrappers,
 Transform("SO PyB/C")))
 env.Depends(cc_file, depends + extra_deps)
-Source(cc_file)
+Source(cc_file, add_tags='python')

 #
 # Handle debug flags
@@ -1218,68 +1279,8 @@
Transform("VER TAGS")))
 env.AlwaysBuild(tags)

-# Embed python files.  All .py files that have been indicated by a
-# PySource() call in a SConscript need to be embedded into the M5
-# library.  To do that, we compile the file to byte code, marshal the
-# byte code, compress it, and then generate a c++ file that
-# 

[gem5-dev] Change in gem5/gem5[develop]: scons: Update the special module importer API.

[Gabe Black (Gerrit) via gem5-dev]

Gabe Black has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/48363 )



Change subject: scons: Update the special module importer API.
..

scons: Update the special module importer API.

In the SConscript, there is a special importer which enables importing
embedded code using various m5.* paths. This was implemented using an
API which has been deprecated and replaced in more recent versions of
python.

Change-Id: I5900f269af48befbcedcb9d25353f04f6297ce9d
---
M src/SConscript
M src/python/importer.py
M src/python/m5/objects/__init__.py
3 files changed, 79 insertions(+), 75 deletions(-)



diff --git a/src/SConscript b/src/SConscript
index dbd28ea..34ba841 100644
--- a/src/SConscript
+++ b/src/SConscript
@@ -42,6 +42,9 @@
 import distutils.spawn
 import functools
 import imp
+import importlib
+import importlib.machinery
+import importlib.util
 import os
 import os.path
 import re
@@ -767,10 +770,20 @@
 #
 SimObject.fixed = True

-class DictImporter(object):
-'''This importer takes a dictionary of arbitrary module names that
-map to arbitrary filenames.'''
+class SimpleModuleLoader(importlib.abc.Loader):
+'''A simple wrapper which delegates setting up a module to a  
function.'''

+def __init__(self, executor):
+super(SimpleModuleLoader, self).__init__()
+self.executor = executor
+def create_module(self, spec):
+return None
+
+def exec_module(self, module):
+self.executor(module)
+
+class M5MetaPathFinder(importlib.abc.MetaPathFinder):
 def __init__(self, modules):
+super(M5MetaPathFinder, self).__init__()
 self.modules = modules
 self.installed = set()

@@ -780,42 +793,46 @@
 del sys.modules[module]
 self.installed = set()

-def find_module(self, fullname, path):
-if fullname == 'm5.defines':
-return self
+def find_spec(self, fullname, path, target=None):
+spec = None

-if fullname == 'm5.objects':
-return self
+# If this isn't even in the m5 package, ignore it.
+if fullname.startswith('m5.'):
+if fullname.startswith('m5.objects'):
+# When imported in this context, return a spec for a dummy
+# package which just serves to house the modules within it.
+# This is subtley different from "import * from m5.objects"
+# which relies on the __init__.py in m5.objects. That in  
turn
+# indirectly relies on the c++ based _m5 package which  
doesn't

+# exist yet.
+if fullname == 'm5.objects':
+dummy_loader = SimpleModuleLoader(lambda x: None)
+spec = importlib.machinery.ModuleSpec(
+name=fullname, loader=dummy_loader,
+is_package=True)
+spec.loader_state = self.modules.keys()

-source = self.modules.get(fullname, None)
-if source is not None and fullname.startswith('m5.objects'):
-return self
+# If this is a module within the m5.objects package,  
return a

+# spec that maps to its source file.
+elif fullname in self.modules:
+source = self.modules[fullname]
+spec = importlib.util.spec_from_file_location(
+name=fullname, location=source.abspath)

-return None
+# The artificial m5.defines subpackage.
+elif fullname == 'm5.defines':
+def build_m5_defines(module):
+module.__dict__['buildEnv'] = dict(build_env)

-def load_module(self, fullname):
-mod = imp.new_module(fullname)
-sys.modules[fullname] = mod
-self.installed.add(fullname)
+spec = importlib.util.spec_from_loader(name=fullname,
+loader=SimpleModuleLoader(build_m5_defines))

-mod.__loader__ = self
-if fullname == 'm5.objects':
-mod.__path__ = fullname.split('.')
-return mod
+# If we're handling this module, write it down so we can unload it
+# later.
+if spec is not None:
+self.installed.add(fullname)

-if fullname == 'm5.defines':
-mod.__dict__['buildEnv'] = dict(build_env)
-return mod
-
-source = self.modules[fullname]
-if source.modname == '__init__':
-mod.__path__ = source.modpath
-mod.__file__ = source.abspath
-
-compiled = compile(open(source.abspath).read(),  
source.abspath, 'exec')

-exec(compiled, mod.__dict__)
-
-return mod
+return spec

 import m5.SimObject
 import m5.params
@@ -826,7 +843,7 @@
 # install the python importer so we can grab stuff from the source
 # tree itself.  We