[gem5-dev] Broken X86 MOVSD_XMM_P Instruction

[Potter, Brandon]

Hello all,

I am running into an issue with an application of mine where I see memory 
corruption. I have tracked the issue down to a value written out by the 
following two instructions:
movsd  0x250dd6(%rip),%xmm0
movups %xmm0,0x48(%rsp)

I suspect that the movsd instruction should zero out part of the %xmm0 
register, but it fails to do so. The unzeroed part is being written onto my 
application's stack and read later on causing a bad address. I have hardware 
and simulator output which seems to support my premise.

Also I stumbled across this after diagnosing the problem: 
src/arch/x86/isa/insts/simd128/floating_point/data_transfer/move.py:294 (def 
macroop MOVSD_XMM_P)

There a comment in the macroop that literally says "# Zero xmmh". I looked 
around a bit at the microops, but it is not clear what I need to do to flash 
half the register. Can anyone provide a suggestion on where I can look for an 
example?

Thanks,
Brandon
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] RFC: Remove Cached Params in Objects

[Potter, Brandon]

Hi all,

I merged changeset 2367198921 which added support for dynamically creating 
Process objects during simulation. This feature is needed to allow applications 
to call the clone system call (which serves to fork a process) in the middle of 
simulation.

Inadvertently, this creates a problem with dynamically creating SimObjects 
which might be new to the simulator.

The Process class uses a SimObject class as a base class. Both Process and 
SimObject use Params (normally set by Python) to initialize themselves. During 
clone, I duplicate the ProcessParams and pass them into the Process constructor 
while dynamically allocating an instance of a Process object. After I 
initialized ProcessParams and use them to create the new Process object, I 
deleted them (not wanting to leak memory). However, the SimObject class caches 
a const pointer to Params which can cause memory corruption after the free if 
the pointer is accessed. (Jason points this out with 
https://gem5-review.googlesource.com/c/public/gem5/+/18068.)

The issue can be avoided by removing the cached Params pointer and instead only 
storing the relevant fields which are accessed by the class. However, SimObject 
establishes an accessor API for Params which other classes have also 
implemented. The derived classes essentially implement the same method with a 
cast to convert the Params to the derived class type.

This leaves me in a bit of a bind since I am not sure what the established 
design principle is regarding Params. Conceptually, there is no reason why we 
could not store the relevant fields which each of the Params types provide in 
the respective classes. However, it is a fairly extensive change which could be 
contentious.

Does anyone have a suggestion on how to go about resolving the issue?

Regards,
Brandon
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] Kokoro Invocation Logs

[Potter, Brandon]

Hello all,

The kokoro invocation logs need to include simout and simerr logs for 
individual test failures. The source.cloud.google.com hyperlink below shows an 
example where I had some test failure with no indication why the tests failed. 
It is insufficient for us to try to run the tests locally, because we cannot 
build them on our internal cluster. Our IT does not allows us to run docker 
instances so I cannot build the some of the tests that are part of the tester.

https://source.cloud.google.com/results/invocations/6ff06bff-8ff7-4a07-b8cf-c951064ed2cf/log

Can someone responsible for the tester please add a mechanism to view the 
standard simout and simerr output? Alternatively, can someone revise the tester 
Makefiles to remove Docker dependencies?

Regards,
Brandon
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] Syscall_Emulation for Shared Memory

[Potter, Brandon]

Hi Maryam,

You need to tell the simulator how many thread contexts it needs at the 
beginning of execution. There's no thread scheduler in SE mode like there'd be 
in a real operating system. Since clone allows you to dynamically create a lot 
of processes, they need to be bound to thread contexts. Since the simulator 
doesn't know how many the application needs you need to tell it.

Pass in the -n option to the configs/example/se.py script. You could try 
something like -n4 or w/e.

Regards,
Brandon

-Original Message-
From: gem5-dev  On Behalf Of Maryam Babaie
Sent: Monday, March 18, 2019 1:56 PM
To: gem5 Developer List 
Subject: Re: [gem5-dev] Syscall_Emulation for Shared Memory

Hi all,

I tried a simple program for testing clone system call on gem5 (SE mode, X86). 
Eventhough clone is implemented in the list of system calls, via 
cloneFunc; but it works like it's not implemented!
As I checked clone system call always returns in the if clause at
src/sim/syscall_emul.hh:1519 :

if (!(ctc = p->findFreeContext())) {
DPRINTF_SYSCALL(Verbose, "clone: no spare thread context in system"
"[cpu %d, thread %d]", tc->cpuId(), tc->threadId());
return -EAGAIN;
}

Obviously, the reason it returns originates from "ctc =
p->findFreeContext()", in  which "findFreeContext()" returns nullptr, 
p->but I
cant' figure out why it happens.
I appreciate if any one shares their insight in this regard.

Bests,
MB

On Mon, Mar 11, 2019 at 3:39 PM Potter, Brandon 
wrote:

> Hi Maryam,
>
> The code that you are looking at is going to change soon. There is a 
> changeset on the google reviewboard which will add some fidelity to 
> the tracing of mmap/munmaps from the perspective of SE mode ( 
> https://gem5-review.googlesource.com/c/public/gem5/+/12307). Feel free 
> to review it if it's interesting to you.
>
> 1) I don't understand what you are asking.
> 2) The new patch has an implementation.
> 3) We could probably add an implementation that flushes file-backed 
> mmaps to the host file. I have never run into a situation where this 
> was needed in an application that was being simulated (so it doesn't 
> have an implementation).
> 4) It turns out that fork is not used in Linux to create a new 
> process. It uses the clone system call instead. (You can verify this 
> by using the Linux strace command with your application. You should 
> only see a clone call.) You should take a look at the clone implementation.
>
> Regards,
> Brandon
>
> -Original Message-
> From: gem5-dev  On Behalf Of Maryam Babaie
> Sent: Friday, March 8, 2019 4:06 PM
> To: m5-...@m5sim.org
> Subject: [gem5-dev] Syscall_Emulation for Shared Memory
>
> Hi all,
> I have some questions regarding the system calls available in SE mode 
> on
> gem5 for X86. I appreciate if any one shares their insight about them:
>
> 1. For the mmap syscall, the function "mmapImpl" has been implemented.
> Up-to which extend this function will work properly for mmap syscall?
>
> 2. For munmap syscall, they have put the munmapFunc, but the body of 
> the corresponding function is actually empty. So, does this mean it is 
> actually not supported on gem5?
>
> 3. msync syscall is flagged as unimplementedFunc. So, does this mean 
> that mmap will still work but the writes will not be reflected 
> persistently in the region?
>
> 4. Since the fork syscall is also flagged as unimplementedFunc , how 
> is the concept of Copy on Write (generally shared pages) able to be 
> simulated and handled in gem5?
>
> Regards
> ___
> gem5-dev mailing list
> gem5-dev@gem5.org
> http://m5sim.org/mailman/listinfo/gem5-dev
> ___
> gem5-dev mailing list
> gem5-dev@gem5.org
> http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] Syscall_Emulation for Shared Memory

[Potter, Brandon]

Hi Maryam,

The code that you are looking at is going to change soon. There is a changeset 
on the google reviewboard which will add some fidelity to the tracing of 
mmap/munmaps from the perspective of SE mode 
(https://gem5-review.googlesource.com/c/public/gem5/+/12307). Feel free to 
review it if it's interesting to you.

1) I don't understand what you are asking.
2) The new patch has an implementation.
3) We could probably add an implementation that flushes file-backed mmaps to 
the host file. I have never run into a situation where this was needed in an 
application that was being simulated (so it doesn't have an implementation).
4) It turns out that fork is not used in Linux to create a new process. It uses 
the clone system call instead. (You can verify this by using the Linux strace 
command with your application. You should only see a clone call.) You should 
take a look at the clone implementation.

Regards,
Brandon

-Original Message-
From: gem5-dev  On Behalf Of Maryam Babaie
Sent: Friday, March 8, 2019 4:06 PM
To: m5-...@m5sim.org
Subject: [gem5-dev] Syscall_Emulation for Shared Memory

Hi all,
I have some questions regarding the system calls available in SE mode on
gem5 for X86. I appreciate if any one shares their insight about them:

1. For the mmap syscall, the function "mmapImpl" has been implemented.
Up-to which extend this function will work properly for mmap syscall?

2. For munmap syscall, they have put the munmapFunc, but the body of the 
corresponding function is actually empty. So, does this mean it is actually not 
supported on gem5?

3. msync syscall is flagged as unimplementedFunc. So, does this mean that mmap 
will still work but the writes will not be reflected persistently in the region?

4. Since the fork syscall is also flagged as unimplementedFunc , how is the 
concept of Copy on Write (generally shared pages) able to be simulated and 
handled in gem5?

Regards
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] Gem5 Compile issue : Commit : e02ec0c24d56bce4a0d8636a340e15cd223d1930

[Potter, Brandon]

Hello all,

The hsail-x86 target will compile with the following changes. I tried running 
the gpu-hello program with the build and it prints "lelellelelelel" instead of 
"hello*". FWIW, the code compiles now which is an improvement.

https://gem5-review.googlesource.com/c/public/gem5/+/12107
https://gem5-review.googlesource.com/c/public/gem5/+/12108

We are currently in the process of making the GCN3 implementation public which 
will supplant the hsail-x86 target entirely. The newer code should appear in 
the gem5 source within the next few weeks.

Regards,
Brandon

P.S. Buy EPYC instead of Xeon.

-Original Message-
From: Gutierrez, Anthony 
Sent: Thursday, August 16, 2018 1:24 PM
To: gem5 Developer List 
Cc: Potter, Brandon 
Subject: RE: [gem5-dev] Gem5 Compile issue : Commit : 
e02ec0c24d56bce4a0d8636a340e15cd223d1930

Did the recent changes break that? Last time I pushed to the public and ran the 
tester gpu-hello worked. I'm guessing whichever change broken this was not ever 
tested using the regression tester?

HSAIL is deprecated meaning we no longer support it, but that said it should 
still work. We plan on trying to push our GCN3 model to the master branch, 
which is what removes HSAIL and we'd prefer to remove it from master that way, 
but as you know there are issues with that currently. We have some problematic 
changes to SE mode that need to be fixed up, as they are not likely to pass 
public review, although I believe they are functionally correct.

Brandon may have a better idea of what needs to be done.

-Original Message-
From: gem5-dev  On Behalf Of Jason Lowe-Power
Sent: Thursday, August 16, 2018 10:05 AM
To: gem5 Developer List 
Subject: Re: [gem5-dev] Gem5 Compile issue : Commit : 
e02ec0c24d56bce4a0d8636a340e15cd223d1930

Hey Tony,

If HSAIL is deprecated, should we remove it from the public repo?

Also, from the nightly regressions the GPU test is broken:

scons: *** [build/HSAIL_X86/gpu-compute/hsail_code.do] Error 1
> scons: *** [build/HSAIL_X86/arch/hsail/gpu_decoder.do] Error 1
> scons: *** [build/HSAIL_X86/gpu-compute/hsail_code.fo] Error 1


Though, we've all been ignoring the emails from zizzer for a while now.
Which reminds me, I would love some help porting tests over to the new 
infrastructure that will (hopefully) resolve these issues since it's easier to 
use.

Cheers,
Jason

On Thu, Aug 16, 2018 at 9:55 AM Gutierrez, Anthony < anthony.gutier...@amd.com> 
wrote:

> Sampad, are you trying to run our GPU model? If so, HSAIL is deprecated.
> We released an updated version of our GPU simulator at
> gem5.googlesource.com/amd/gem5 branch agutierr/master-gcn3-staging. I 
> recommend you starting with that.
>
> I don’t think anybody at AMD will have time to look into HSAIL bugs 
> for the time being, but doesn’t the current tester test at least 1 GPU 
> program (gpu hello)? How did the changes get by the tester?
>
> If you encounter issues with our GCN3 model, you can notify us through 
> the
> gem5 mailing list.
>
> From: Jason Lowe-Power 
> Sent: Thursday, August 16, 2018 9:46 AM
> To: gem5 Developer List ; Gutierrez, Anthony < 
> anthony.gutier...@amd.com>
> Subject: Re: [gem5-dev] Gem5 Compile issue : Commit :
> e02ec0c24d56bce4a0d8636a340e15cd223d1930
>
> Hi Sampad,
>
> It looks like you've found a hole in our testing ;). We don't compile 
> HSAIL regularly, and it looks like some bugs have worked their way in.
> Tony, do you think you, or someone else that's familiar with the GPU 
> model could take a look?
>
> Cheers,
> Jason
>
> On Thu, Aug 16, 2018 at 8:36 AM Sampad Mohapatra  su...@psu.edu>> wrote:
> Hi Gabe,
>
> I am building on a 64 bit cluster.
>
> I might have found the reasons for the issues:
>
> (1) There is no function "const_iterator find(Addr r) const" in 
> src/base/addr_range_map.hh.
>
> (2) In src/base/types.hh, struct TypedAtomicOpFunctor has a pure 
> virtual function "clone()". This struct is the base for a
>
>  lot of GPU op classes in src/gpu-compute/gpu_dyn_inst.hh. But 
> these classes don't define clone(). This results in
>  the errors I mentioned in (2).
>
> Sampad
>
> On Tue, Aug 14, 2018 at 2:26 PM Gabe Black wrote:
> > I wonder if you're building on a 32 bit machine? It could be that
> somebody
> > made an assumption about how big certain types are, and some 
> > function definitions which normally turn out to be the same thing 
> > are different if certain types are no longer equivalent. That would 
> > be my gut reaction to the second problem. Maybe add "override" in 
> > places where a virtual
> function
> > is being overridden in the GPU code related to the error? I bet the 
> > compiler complains about at least one of them.
>
> &

[gem5-dev] scons configuration run only once

[Potter, Brandon]

Hello everyone,

I would like to rewrite some of the scons infrastructure so that the 
configuration tests are executed only once upon first building the source. This 
differs from the current situation where they're run every time anything is 
done. Currently, you cannot even print help information without the scons 
configuration checks running. Running them every time anything is done seems 
unnecessary to me.

Anyone else have an opinion on the topic?

Regards,
Brandon
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] [EXT] Re: testing framework

[Potter, Brandon]

Is there any documentation on how to run the current unit tests? I notice that 
there's a unit test directory in the X86 build folder, but nothing inside is 
executable. I do not see a way of generating output from SCons related to 
unittests either.

From the src/SConscript file, I see that the unittests should get created 
inside the makeEnv function, but it doesn't appear that this does anything (at 
least for X86).

-Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Sandberg
Sent: Wednesday, November 22, 2017 3:35 AM
To: gem5 Developer List <gem5-dev@gem5.org>; Gabe Black <gabebl...@google.com>
Subject: Re: [gem5-dev] [EXT] Re: testing framework

One issue that is preventing me from running the unit tests is that they 
require separate scons targets. One solution that I was considering at one 
point was to make the unit tests available as Python exports in the main build. 
This would allow us to run a magic config that calls unit tests instead of 
running the simulator proper. I think it would work for
all(?) current unit tests, but I don't know how it would interact with other 
test unit test frameworks.

Cheers,
Andreas

On 21/11/2017 01:37, Gabe Black wrote:
> What I would minimally like to see is for people to run the unit tests 
> that already exist. Clearly that hasn't been happening since one of 
> them hasn't compiled since 2014 (now fixed). We need our tests to test 
> something specific, say whether or not it worked (not leave 
> interpretation up to the reader), and for people to actually run them.
>
> It's not at all clear what to do to actually build the unit tests from 
> scons, or how to run them, or what their results mean. I'm going to 
> try cleaning these up a bit. Also, they shouldn't be something that 
> lives off on the side or in their own directory, out of sight and out 
> of mind, they should be alongside the code they test.
>
> It might even make sense to associate a unit test with a particular 
> source file and make compilation fail if the unit test doesn't pass. 
> That might be a bit overboard, but would at least ensure that the tests get 
> run.
>
> Gabe
>
> On Mon, Nov 20, 2017 at 3:14 PM, Potter, Brandon 
> <brandon.pot...@amd.com>
> wrote:
>
>> I also support adding unit tests (and possibly a code coverage 
>> checker to identify weak points). Ideally, we'd add unit tests for 
>> the entire simulator, but we should probably put something into place 
>> for new code requiring some type of unit tests. Otherwise, only a 
>> subset of the submissions will contain unit tests.
>>
>> Regards,
>> Brandon
>>
>> -Original Message-
>> From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Jason 
>> Lowe-Power
>> Sent: Monday, November 20, 2017 11:22 AM
>> To: gem5 Developer List <gem5-dev@gem5.org>
>> Subject: Re: [gem5-dev] [EXT] Re: testing framework
>>
>> I'm all for unit tests. GTest seems fine, but I'm not a huge fan of 
>> its syntax. Although, that shouldn't stop us from having unit tests!!
>>
>> What's the state of getting whimsy pushed in? From my side, it's 
>> ready to go. The only thing on my to do list that I would have liked 
>> to finish up before pushing is better integration with the current test 
>> infrastructure.
>> It would be good for whoever is running with the old test 
>> infrastructure to be able to run both the old-style and new-style tests with 
>> one command.
>> Unfortunately, I don't have the time to dig into this for the 
>> foreseeable future.
>>
>> Jason
>>
>> On Fri, Nov 17, 2017 at 3:19 PM Paul Rosenfeld (prosenfeld) < 
>> prosenf...@micron.com> wrote:
>>
>>> Even without the complexities of setting up SystemC+gem5 it's pretty 
>>> painful to "unit" test SystemC because the SystemC runtime has a lot 
>>> of global state that comes along with it (for example: you can't "uncall"
>>> sc_start() to go back and re-do elaboration with different modules).
>>> The only (not very elegant) workaround we have discussed for unit 
>>> testing SystemC modules is to just to build each set of related 
>>> SystemC unit tests into their own independent GTest binaries.
>>>
>>> On the whole, though, GTest is pretty great.
>>>
>>> -Original Message-
>>> From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe 
>>> Black
>>> Sent: Friday, November 17, 2017 5:51 PM
>>> To: Matthias Jung <jun...@eit.uni-kl.de>
>>> Cc: gem5 Developer List <gem5-dev@gem5.org>
>>> Subject: [EXT] Re: [gem5-dev] 

Re: [gem5-dev] [EXT] Re: testing framework

[Potter, Brandon]

I also support adding unit tests (and possibly a code coverage checker to 
identify weak points). Ideally, we'd add unit tests for the entire simulator, 
but we should probably put something into place for new code requiring some 
type of unit tests. Otherwise, only a subset of the submissions will contain 
unit tests.

Regards,
Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Jason Lowe-Power
Sent: Monday, November 20, 2017 11:22 AM
To: gem5 Developer List 
Subject: Re: [gem5-dev] [EXT] Re: testing framework

I'm all for unit tests. GTest seems fine, but I'm not a huge fan of its syntax. 
Although, that shouldn't stop us from having unit tests!!

What's the state of getting whimsy pushed in? From my side, it's ready to go. 
The only thing on my to do list that I would have liked to finish up before 
pushing is better integration with the current test infrastructure.
It would be good for whoever is running with the old test infrastructure to be 
able to run both the old-style and new-style tests with one command.
Unfortunately, I don't have the time to dig into this for the foreseeable 
future.

Jason

On Fri, Nov 17, 2017 at 3:19 PM Paul Rosenfeld (prosenfeld) < 
prosenf...@micron.com> wrote:

> Even without the complexities of setting up SystemC+gem5 it's pretty 
> painful to "unit" test SystemC because the SystemC runtime has a lot 
> of global state that comes along with it (for example: you can't "uncall"
> sc_start() to go back and re-do elaboration with different modules). 
> The only (not very elegant) workaround we have discussed for unit 
> testing SystemC modules is to just to build each set of related 
> SystemC unit tests into their own independent GTest binaries.
>
> On the whole, though, GTest is pretty great.
>
> -Original Message-
> From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Gabe 
> Black
> Sent: Friday, November 17, 2017 5:51 PM
> To: Matthias Jung 
> Cc: gem5 Developer List 
> Subject: [EXT] Re: [gem5-dev] testing framework
>
> As far as a high level framework, I think whimsy should help there 
> since it's a bit more decoupled and will let you run arbitrary things 
> as tests, at least as far as I can tell. Since the systemC integration 
> is just that, an integration of two systems, I don't think unit tests 
> would be as applicable generally speaking. You could have one for some 
> things though, like the ports that translate between gem5's and systemC's 
> memory protocol.
>
> GTest claims to be BSD, so the license should be compatible.
>
> Gabe
>
> On Fri, Nov 17, 2017 at 10:14 AM, Matthias Jung 
> wrote:
>
> > Hey,
> >
> > It would be also nice if we have something that we could also use 
> > for test the SystemC stuff. Right now this is not possible with the 
> > current framework. Because we have to build gem5 without python as a 
> > library for the SystemC stuff... and then build again with linking 
> > of
> the gem5 library.
> >
> > Regards
> > Matthias
> >
> > > Am 17.11.2017 um 17:43 schrieb Andreas Sandberg <
> > andreas.sandb...@arm.com>:
> > >
> > > I'm generally in favour of switching to an existing unit test
> framework.
> > > I haven't surveyed the C++ test framework landscape, but judging 
> > > by the users (Chromium, LLVM, ...) GTest seems pretty solid.
> > >
> > > As far as I can tell, GTest is all BSD. Is that correct?
> > >
> > > Cheers,
> > > Andreas
> > >
> > >> On 16/11/2017 05:04, Gabe Black wrote:
> > >> Hi folks. How is the test framework integration coming along? Is 
> > >> there anything I can do to help?
> > >>
> > >> Also, looking at the whimsy documentation and talking to some 
> > >> other
> > folks
> > >> at Google, it looks like the framework is good at running tests, 
> > >> but doesn't itself really implement, for instance, unit tests 
> > >> which have
> > their
> > >> own C++ main function, etc.
> > >>
> > >> Does anyone have an opinion about using the google gtest library 
> > >> for writing C++ unit tests? I've only worked with it a little bit 
> > >> superficially, but so far it seems pretty nice and seems like it 
> > >> would
> > fit
> > >> into whimsy fairly well.
> > >>
> > >> Gabe
> > >> ___
> > >> gem5-dev mailing list
> > >> gem5-dev@gem5.org
> > >> http://m5sim.org/mailman/listinfo/gem5-dev
> > >
> > > IMPORTANT NOTICE: The contents of this email and any attachments 
> > > are
> > confidential and may also be privileged. If you are not the intended 
> > recipient, please notify the sender immediately and do not disclose 
> > the contents to any other person, use it for any purpose, or store 
> > or copy the information in any medium. Thank you.
> > > ___
> > > gem5-dev mailing list
> > > gem5-dev@gem5.org
> > > http://m5sim.org/mailman/listinfo/gem5-dev
> >
> >
> 

Re: [gem5-dev] More problems in SE land

[Potter, Brandon]

I will post a patch to fix the compilation problem in the next few days.

-Brandon

From: Andreas Hansson [mailto:andreas.hans...@arm.com]
Sent: Friday, March 10, 2017 2:29 AM
To: gem5 Developer List <gem5-dev@gem5.org>
Cc: Potter, Brandon <brandon.pot...@amd.com>
Subject: Re: More problems in SE land

Merely a reminder that this is still broken.

Andreas

From: Andreas Hansson <andreas.hans...@arm.com<mailto:andreas.hans...@arm.com>>
Date: Tuesday, 7 March 2017 at 08:54
To: gem5 Developer List <gem5-dev@gem5.org<mailto:gem5-dev@gem5.org>>
Subject: More problems in SE land

Hi all (and Brandon in particular),

More problems in SE land I’m afraid. The following changeset prevents gem5 from 
building on macOS:

commit 2367198921765848a4f5b3d020a7cc5776209f80
Author: Brandon Potter <brandon.pot...@amd.com<mailto:brandon.pot...@amd.com>>
Date:   Mon Feb 27 14:10:15 2017 -0500

syscall_emul: [PATCH 15/22] add clone/execve for threading and multiprocess 
simulations

Modifies the clone system call and adds execve system call. Requires 
allowing
processes to steal thread contexts from other processes in the same system
object and the ability to detach pieces of process state (such as MemState)
to allow dynamic sharing.

The result is the following:

build/ARM/sim/process.cc:157:9: error: use of undeclared identifier 'CLONE_VM'
if (CLONE_VM & flags) {
^
build/ARM/sim/process.cc:187:9: error: use of undeclared identifier
  'CLONE_FILES'
if (CLONE_FILES & flags) {
^
build/ARM/sim/process.cc:227:9: error: use of undeclared identifier
  'CLONE_THREAD'
if (CLONE_THREAD & flags) {
^
3 errors generated.

I am inclined to suggest we create a linux_process.cc or similar for all the 
sys calls that are only available on Linux (to avoid ifdef’s all over the 
place).

Andreas

IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] Migrating to git and gerrit

[Potter, Brandon]

Hello Andreas,

Newbie gerrit/git questions here:

If someone requests a modification to a changeset in a review (or I realize 
something is wrong with the changeset myself), how do I update the posted 
review on Gerrit for the specified changeset? Mercurial/Reviewboard would be 
something like hg qref followed by hg postreview -oue  and the patch on 
Reviewboard would be updated to reflect the changes. How do we achieve a 
similar behavior here?

Furthermore, what does the resulting commit log look like given whatever 
workflow you use? Ideally, we'd like the commit log to reflect coherent, atomic 
changes that implement one specific feature at a time. I don't want to public 
many incoherent changesets that are iterating to the right/accepted solution to 
implement a feature.

Regards,
Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Sandberg
Sent: Wednesday, March 1, 2017 8:44 AM
To: gem5 Developer List 
Subject: Re: [gem5-dev] Migrating to git and gerrit

Hi Everyone,

The new Gerrit-based infrastructure is now live with the old Mercurial 
repository as a read-only mirror. Mirroring is currently done using a cron job 
that executes every 15 minutes, which means that there is some amount of lag 
between the gerrit master version and the mirrors.

Those of you with commit access who plan to submit patches that have been on 
reviewed using the old infrastructure will need to convert these patches to git 
and push them to the main repository. It is highly recommended that patches 
without reviews are discarded and re-posted using the Gerrit flow to reduce the 
maintenance overhead of committing patches.

Existing Mercurial patches (e.g., from patch queues) can be converted using the 
hg-patch-to-git-patch [1] script from Mozilla. The patch conversion script 
makes sure that the commit message and author information can be understood by 
git. Before converting your patches, make sure they apply cleanly on the 
current master branch. The best way to check this is by qpop:ing your patch 
queue, pulling in new changes, and then reapplying it. If you get warnings when 
applying patches, you need to qrefresh them to make sure that the diffs are up 
to date.
Failing to do so will result in git refusing the patch.

Once you have a patch queue that applies cleanly, you can import it into a new 
git branch using the following commands:

# Create a new clone of the master repository git clone 
https://gem5.googlesource.com/public/gem5 gem5 cd gem5 # Install Gerrit's 
commit message hook. This is done automatically by scons when you build gem5.
cp ext/git-commit-msg .git/hooks/commit-msg

# Create a new branch for the outgoing changes git checkout -b fixes 
origin/master

hg-patch-to-git-patch /path/to/old/gem5/.hg/patches/patch1.patch | git am # Run 
the commit message hook to generate a Change-id by amending the commit git 
commit --amend ...
hg-patch-to-git-patch /path/to/old/gem5/.hg/patches/patchN.patch | git am git 
commit --amend

To push the branch for review (remember to test it and verify that the commit 
log looks OK first!), execute the following command:

git push origin HEAD:refs/for/master

It's sometimes useful to set a patch series topic if you're submitting multiple 
patches. To do that when you submit patches for review, use this command 
instead:

git push origin HEAD:refs/for/master%topic=my/topic

Some users can bypass reviews and push changes straight into the master branch. 
This should only be done for code that has been reviewed using the 
ReviewBoard-based flow. If you're one of the select few, you can use the 
following push command to bypass review:

git push origin HEAD:refs/heads/master


Cheers,
Andreas

[1]
https://github.com/mozilla/moz-git-tools/raw/master/hg-patch-to-git-patch


On 16/02/2017 16:54, Jason Lowe-Power wrote:
> Hi all,
>
> We've been talking about this for a while, but now it's time! Special 
> thanks to Andreas Sandberg for all of his hard work for putting this 
> together.
>
> We will be migrating our infrastructure from the self-hosted mercurial 
> repo at repo.gem5.org and reviewboard to git and gerrit hosted on 
> Google's new googlesource website. You can find a live version (not 
> ready for primetime) of this at https://gem5.googlesource.com/.
>
> We are planning on flipping the switch on *March 1st*, unless there is 
> an objection from the community. Note: I'm announcing this on gem5-dev 
> before announcing on gem5-users and gem5-announce in case there's any 
> details we've missed.
>
> I've posted a patch on reviewboard that contains a new CONTRIBUTING 
> document that details the new contribution process. Please review that 
> document so it can be pushed before we transition. (
> http://reviews.gem5.org/r/3814/)
>
> The major changes are detailed below:
> 1. REPOSITORIES
>* The canonical version of gem5 will now live at 
> https://gem5.googlesource.com/, not repo.gem5.org

Re: [gem5-dev] ISA Deprecation

[Potter, Brandon]

Hi Jason,

Thanks for the links; I was not aware that they existed.

I am not going to try to press the issue if others want to keep the ISAs. I was 
not sure what the consensus was on keeping them, but it seems that some folks 
want them to stick around.

Regards,
Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Jason Lowe-Power
Sent: Monday, February 27, 2017 10:24 AM
To: gem5 Developer List <gem5-dev@gem5.org>
Subject: Re: [gem5-dev] ISA Deprecation

Hi Brandon,

See this discussion for some of the history here:
http://comments.gmane.org/gmane.comp.emulators.m5.devel/30510

Let me try to summarize for everyone.

There are a number of reasons to deprecate/phase out most of the supported 
ISAs. Briefly:
 - Less work for contributors especially when updating SE mode
 - Less testing time
 - Some ISAs have no maintainer
 - Some ISAs have (basically) no tests, or the tests are proprietary
 - Causes confusion for users (e.g., they run experiments with an incomplete 
ISA, or worse)

However, some people *do* use the code for ISAs other than x86, ARM, RISC-V, 
and HSAIL. For instance:
 - From Jakub "[gem5 is] the only existing and basically working open source 
sun4v simulator.
 - Boris uses gem5's support for MIPS and POWER to do retargetable compiler 
research, and "dropping them would be pretty much equivalent to discontinuing 
GEM5 altogether"

I think Steve makes one of the best arguments both for and against this in this 
message: http://www.mail-archive.com/gem5-dev@gem5.org/msg19429.html.

-- My opinions below:

From our discussion at HPCA, I think it comes down to "what does the community 
want gem5 to be?" Right now, gem5 has a wide variety of different use cases. I 
think we need to decide as a community what use cases we want to support and 
focus our limited developer time on those use cases. For instance, do we think 
it is worth it to support MIPS emulation?

Maybe someone can come up with a list of "gem5 use cases" for us to look at and 
discuss. I don't have time in the next week or so, but after that I'll put it 
on my to do list.

At a higher level, I strongly believe we need to take a step back and make sure 
that gem5 is serving its users as best as it can. There's a lot of unmaintained 
code in gem5 that is hurting and not helping. For instance, it would be great 
if Brandon could focus his time purely on improving x86 SE mode, which many 
people use, and not fixing bugs with SPARC.

Cheers,
Jason

On Thu, Feb 23, 2017 at 4:47 PM Potter, Brandon <brandon.pot...@amd.com>
wrote:

Hello all,

A colleague mentioned that ISA deprecation was discussed during the recent
gem5 meeting. I am wondering what the community's feelings are toward the idea 
and which ISAs would be on the chopping block. Personally, I'd like to kill 
ALPHA, MIPS, POWER, and SPARC. This means that we'd retain X86, ARM, RISCV, and 
HSAIL-X86.

I can help out with the removal if we decide that we want to kill some of them. 
We might wait until March 1st to start the process since we're supposed to 
transition to fully transition to Git on that day; we could create a tag for 
the ISA deprecation in case someone wants to revive one of them in the future 
(i.e. MIPS or POWER if someone wants to maintain them).

Regards,
Brandon


___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] Linux Test Project - Testing SE Mode

[Potter, Brandon]

Hello all,

https://github.com/linux-test-project/ltp/tree/master/testcases/kernel/syscalls

I was considering writing Linux test cases for SE Mode to work out (potential 
but more likely probable) bugs in the implementation of some of the system 
calls, but stumbled across a source base that already has tests for Linux. What 
is the process for getting tests added to the regression system for the new git 
repository? (If the information already exists, I apologize. Please point me to 
the right documentation.)

We only support a subset of the system calls that are tested for in LTP. So 
ideally, we could pull the source, build it, and choose a subset of the tests 
that we'd need. As other syscalls are implemented in gem5 in the future, we can 
add the LTP tests into the regressions to test them without needing to worry 
about always adding in our own.

I am a little concerned about getting cross-compiled versions of the binaries 
so that we can test each ISA. The build machine would need a cross-compiler to 
actually guarantee that the ISAs have coverage. (I am not sure that the 
compilers even support ALPHA although maybe they still do.)

I believe that the license is GPLv2 for LTP, but we are not including it into 
our code so I do not think that it matters; it would just be a regression 
target for SE Mode.  If this actually matters, someone should speak up.

Regards,
Brandon

___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] ISA Deprecation

[Potter, Brandon]

Hello all,

A colleague mentioned that ISA deprecation was discussed during the recent gem5 
meeting. I am wondering what the community's feelings are toward the idea and 
which ISAs would be on the chopping block. Personally, I'd like to kill ALPHA, 
MIPS, POWER, and SPARC. This means that we'd retain X86, ARM, RISCV, and 
HSAIL-X86.

I can help out with the removal if we decide that we want to kill some of them. 
We might wait until March 1st to start the process since we're supposed to 
transition to fully transition to Git on that day; we could create a tag for 
the ISA deprecation in case someone wants to revive one of them in the future 
(i.e. MIPS or POWER if someone wants to maintain them).

Regards,
Brandon


___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] Thoughts On Project Using C++14

[Potter, Brandon]

Hello all,

What is the general feeling behind bumping to C++14 instead of keeping the 
top-level SConstruct flag at C++11? I would like to use generic lambdas in some 
of my code, but cannot do so because it's not available until C++14.

Both GCC and clang support a full implementation of C++14 now: GCC became fully 
compliant as of GCC 6.1 (04/27/16); clang was compliant with clang 3.4 
(01/01/14).

https://gcc.gnu.org/projects/cxx-status.html#cxx14
https://gcc.gnu.org/gcc-6/
https://clang.llvm.org/cxx_status.html
http://releases.llvm.org

Regards,
Brandon
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] Open Review Request - 3806

[Potter, Brandon]

Hello all,

I have a review open to fix a bug for the SPARC TLB and translation code that 
only affects SE Mode. The fix should solve the SE Mode regression issue where 
the SPARC tests run forever. I posted it a little over a week ago and plan to 
push it this coming Thursday (02/16/17) if I do not get any new reviews in.

http://reviews.gem5.org/r/3806/diff/2/#index_header

Thanks,
Brandon
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] changeset in gem5: syscall_emul: [patch 4/22] remove redundant M...

[Potter, Brandon]

Yes, I'm looking into this; this is priority #1 for me right now.

-Brandon

-Original Message-
From: Andreas Sandberg [mailto:andreas.sandb...@arm.com] 
Sent: Friday, January 27, 2017 6:40 AM
To: gem5 Developer List <gem5-dev@gem5.org>; gem5-...@m5sim.org; Potter, 
Brandon <brandon.pot...@amd.com>
Subject: Re: [gem5-dev] changeset in gem5: syscall_emul: [patch 4/22] remove 
redundant M...

Hi Brandon,

Something bad (they hang) happened with the SE-mode ALPHA and SPARC regressions 
the other day. A quick bisection suggest that this commit was the first bad 
one. Could you have a look?

//Andreas

On 23/01/17 21:22, Brandon Potter wrote:
> changeset cd7f3a1dbf55 in /z/repo/gem5
> details: http://repo.gem5.org/gem5?cmd=changeset;node=cd7f3a1dbf55
> description:
>   syscall_emul: [patch 4/22] remove redundant M5_pid field from 
> process
>
> diffstat:
>
>   src/arch/alpha/process.cc |   4 ++--
>   src/arch/sparc/faults.cc  |   4 ++--
>   src/sim/Process.py|  13 +++--
>   src/sim/process.cc|  15 ++-
>   src/sim/process.hh|  38 ++
>   src/sim/system.cc |   3 ---
>   src/sim/system.hh |   8 
>   7 files changed, 35 insertions(+), 50 deletions(-)
>
> diffs (243 lines):
>
> diff -r 54436a1784dc -r cd7f3a1dbf55 src/arch/alpha/process.cc
> --- a/src/arch/alpha/process.cc   Wed Nov 09 14:27:40 2016 -0600
> +++ b/src/arch/alpha/process.cc   Wed Nov 09 14:27:40 2016 -0600
> @@ -179,7 +179,7 @@
>   AlphaLiveProcess::setupASNReg()
>   {
>   ThreadContext *tc = system->getThreadContext(contextIds[0]);
> -tc->setMiscRegNoEffect(IPR_DTB_ASN, M5_pid << 57);
> +tc->setMiscRegNoEffect(IPR_DTB_ASN, _pid << 57);
>   }
>
>
> @@ -187,7 +187,7 @@
>   AlphaLiveProcess::loadState(CheckpointIn )
>   {
>   LiveProcess::loadState(cp);
> -// need to set up ASN after unserialization since M5_pid value may
> +// need to set up ASN after unserialization since _pid value may
>   // come from checkpoint
>   setupASNReg();
>   }
> diff -r 54436a1784dc -r cd7f3a1dbf55 src/arch/sparc/faults.cc
> --- a/src/arch/sparc/faults.ccWed Nov 09 14:27:40 2016 -0600
> +++ b/src/arch/sparc/faults.ccWed Nov 09 14:27:40 2016 -0600
> @@ -630,7 +630,7 @@
>   } else {
>   Addr alignedVaddr = p->pTable->pageAlign(vaddr);
>   tc->getITBPtr()->insert(alignedVaddr, 0 /*partition id*/,
> -p->M5_pid /*context id*/, false, entry.pte);
> +p->_pid /*context id*/, false, entry.pte);
>   }
>   }
>
> @@ -654,7 +654,7 @@
>   } else {
>   Addr alignedVaddr = p->pTable->pageAlign(vaddr);
>   tc->getDTBPtr()->insert(alignedVaddr, 0 /*partition id*/,
> -p->M5_pid /*context id*/, false, entry.pte);
> +p->_pid /*context id*/, false, entry.pte);
>   }
>   }
>
> diff -r 54436a1784dc -r cd7f3a1dbf55 src/sim/Process.py
> --- a/src/sim/Process.py  Wed Nov 09 14:27:40 2016 -0600
> +++ b/src/sim/Process.py  Wed Nov 09 14:27:40 2016 -0600
> @@ -43,6 +43,13 @@
>   kvmInSE = Param.Bool('false', 'initialize the process for KvmCPU in SE')
>   max_stack_size = Param.MemorySize('64MB', 'maximum size of the 
> stack')
>
> +uid = Param.Int(100, 'user id')
> +euid = Param.Int(100, 'effective user id')
> +gid = Param.Int(100, 'group id')
> +egid = Param.Int(100, 'effective group id')
> +pid = Param.Int(100, 'process id')
> +ppid = Param.Int(99, 'parent process id')
> +
>   @classmethod
>   def export_methods(cls, code):
>   code('bool map(Addr vaddr, Addr paddr, int size, bool 
> cacheable=true);') @@ -60,12 +67,6 @@
>   cmd = VectorParam.String("command line (executable plus arguments)")
>   env = VectorParam.String([], "environment settings")
>   cwd = Param.String('', "current working directory")
> -uid = Param.Int(100, 'user id')
> -euid = Param.Int(100, 'effective user id')
> -gid = Param.Int(100, 'group id')
> -egid = Param.Int(100, 'effective group id')
> -pid = Param.Int(100, 'process id')
> -ppid = Param.Int(99, 'parent process id')
>   simpoint = Param.UInt64(0, 'simulation point at which to start 
> simulation')
>   drivers = VectorParam.EmulatedDriver([], 'Available emulated 
> drivers')
>
> diff -r 54436a1784dc -r cd7f3a1dbf55 src/sim/process.cc
> --- a/src/sim/process.cc  Wed Nov 09 14:27:40 2016 -0600
> +++ b/src/sim/process.cc  Wed Nov 09 14:27:40 2016 -0600
> @@ -131,12 +131,11 @@
> brk_point(

Re: [gem5-dev] Compilation error for gem5 after statfs change

[Potter, Brandon]

Hello all,

I'm trying to resolve this issue. I'm setting up FreeBSD 11 on a VirtualBox 
instance now to see if I can resolve the problems. Hopefully this covers OSX 
indirectly as well.

I don't expect that moving the offending system calls into Linux specific files 
will resolve the problem where the underlying host OS needs to make the system 
call on behalf of the target (like what happens with fallocate). I think the 
only way to resolve the issue is to add in preprocessor directives to prevent 
compilation/linkage errors on platforms that don't support what's necessary 
under the covers. (Correct me if there's a better way to handle this.)

I'll post a patch and respond to the mailing list when I have it working.

Regards,
Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Hansson
Sent: Thursday, January 26, 2017 2:18 AM
To: gem5 Developer List 
Subject: Re: [gem5-dev] Compilation error for gem5 after statfs change

Hi all,

While I agree about the end goal (test every commit on every platform) I would 
suggest we go about this one step at a time. The likelihood of breakage is only 
high for anything related to syscall emulation, so it really only affects a 
small part of the developer community. Agreed?

With that in mind I think we can prevent 99% of the issues if syscall emulation 
patches are built on a BSD system with clang. I would be wrong, but I would 
think this is a very good filter, with a fairly low cost and effort of 
deployment. Virtualbox + some flavour of BSD would do it.

Andreas

On 25/01/2017, 23:18, "gem5-dev on behalf of Jason Lowe-Power"
 wrote:

>Yeah, it's a major problem that we say that we support macOS and others 
>when we allow commits that break builds on these other OSes.
>
>If we are going to say that we have support for OSes other than Linux, 
>we need to at least verify gem5 builds on these OSes, preferably before 
>accepting a commit. I'm currently testing out the free Travis-CI 
>service ( https://travis-ci.org/powerjg/gem5-ci-test). We could 
>probably hook this up to our gem5 github page, if it works out.
>
>Another important point, though, is that we can't expect all committers 
>to own multiple machines to test their changes on. We need something 
>that will do pre-commit builds on all the platforms we claim to 
>support.
>
>We're in the middle of moving the regression tests to a hosted jenkins 
>instance. Hopefully this will solve some of these issues (though I 
>don't think it will support multiple OS builds).
>
>Do others have any ideas on a long-term solution here?
>
>Cheers,
>Jason
>
>On Tue, Jan 24, 2017 at 5:46 PM Bjoern A. Zeeb < 
>bzeeb-li...@lists.zabbadoz.net> wrote:
>
>On 24 Jan 2017, at 22:08, Jason Lowe-Power wrote:
>
>> Hi Brandon,
>>
>> I think this is a "real" bug:
>>
>http://qa.gem5.org//1905/compiling-problem-gem5-mac-os-10-11-6-scons-bu
>ild
>-arm-gem5-opt
>.
>> I think there are a few more places that need an #ifdef NO_STATFS.
>> Could
>> you look into it and post a patch if there's a problem? If not, 
>> please reply to the gem5 QA post and let them know.
>
>Can people try this one and probably get the #ifdefs right for NetBSD 
>as well?  There are at least 3 different checks for that code;  if 
>people don’t care about “style” I could get it right, but ..
>
>diff -r e47703369039 src/sim/syscall_emul.hh
>--- a/src/sim/syscall_emul.hh   Fri Jan 20 14:12:58 2017 -0500
>+++ b/src/sim/syscall_emul.hh   Tue Jan 24 23:45:04 2017 +
>@@ -70,6 +70,8 @@
>  #include 
>  #if (NO_STATFS == 0)
>  #include 
>+#else
>+#include 
>  #endif
>  #include 
>  #include 
>@@ -530,20 +532,25 @@
>  {
>  TypedBufferArg tgt(addr);
>
>+tgt->f_type = TheISA::htog(host->f_type);
>  #if defined(__OpenBSD__) || defined(__APPLE__) || defined(__FreeBSD__)
>-tgt->f_type = 0;
>+tgt->f_bsize = TheISA::htog(host->f_iosize);
>  #else
>-tgt->f_type = TheISA::htog(host->f_type);
>+tgt->f_bsize = TheISA::htog(host->f_bsize);
>  #endif
>-tgt->f_bsize = TheISA::htog(host->f_bsize);
>  tgt->f_blocks = TheISA::htog(host->f_blocks);
>  tgt->f_bfree = TheISA::htog(host->f_bfree);
>  tgt->f_bavail = TheISA::htog(host->f_bavail);
>  tgt->f_files = TheISA::htog(host->f_files);
>  tgt->f_ffree = TheISA::htog(host->f_ffree);
>  memcpy(>f_fsid, >f_fsid, sizeof(host->f_fsid));
>+#if defined(__OpenBSD__) || defined(__APPLE__) || defined(__FreeBSD__)
>+tgt->f_namelen = TheISA::htog(host->f_namemax);
>+tgt->f_frsize = TheISA::htog(host->f_bsize); #else
>  tgt->f_namelen = TheISA::htog(host->f_namelen);
>  tgt->f_frsize = TheISA::htog(host->f_frsize);
>+#endif
>  memcpy(>f_spare, >f_spare, sizeof(host->f_spare));
>
>  tgt.copyOut(mem);
>___
>gem5-dev mailing list
>gem5-dev@gem5.org
>http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] FW: [gem5-users] scons build error for Garnet 2.0

[Potter, Brandon]

Well, it's good that someone else has similar feelings, but I feel that Alpha 
is more of a drag on development than Power or MIPS. I've run into several 
issues where I'd like to change a system call to be templated to give it more 
functionality and then I run into the issue of needing to a) keep the old 
non-templated version around for operating system where I cannot figure out the 
flags; b) search in dark places on the internet for said flags; c) make up 
values for said flags to make the compiler stop complaining. I haven't done c) 
yet with anything other than hacks, but it seems more and more appealing. At 
least with Linux, I can pull up the source, examine it, and try to reason about 
what the values for flags will be even if I can't run test programs to verify.

I don't know how anyone could actually use Tru64 or Alpha for research; one 
eventually needs to compile new applications and that seems like it would 
require a machine with the Alpha ISA (and Tru64). Cross compiling especially 
for older architectures is a headache so I'm at a loss as to why anyone would 
want to even bother with it.

-Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Jason Lowe-Power
Sent: Monday, October 17, 2016 9:34 AM
To: gem5 Developer List <gem5-dev@gem5.org>
Subject: Re: [gem5-dev] FW: [gem5-users] scons build error for Garnet 2.0

Hi Brandon,

We've had this discussion before. See this thread:
http://www.mail-archive.com/gem5-dev@gem5.org/msg19380.html

In summary... I think we're planning on phasing out ALPHA, but that won't 
happen in the very short term (weeks), but may happen in the medium term 
(months).

Cheers,
Jason

On Mon, Oct 17, 2016 at 9:28 AM Potter, Brandon <brandon.pot...@amd.com>
wrote:

> Switching this over to the dev mailing list since it seems more 
> suitable (and I don’t have a user account; need to rectify that at some 
> point).
>
> -Brandon
>
> From: Potter, Brandon
> Sent: Monday, October 17, 2016 9:25 AM
> To: 'Andreas Hansson' <andreas.hans...@arm.com>; gem5 users mailing 
> list < gem5-us...@gem5.org>
> Subject: RE: [gem5-users] scons build error for Garnet 2.0
>
> Yes, I can fix it, but it’s difficult to know what values are for Tru64.
>
> Can we deprecate Alpha already?  It looks like the last commercial 
> product that came out with Alpha was in 2007, almost a decade ago. I 
> don’t think that I know anyone personally who uses a 10 year old 
> computer.  I don’t think that I’ve ever used an Alpha either to boot; 
> wondering why I need to go out of my way to try to support it.
>
> Does anyone use Alpha outside of running our regressions?
>
> -Brandon
>
> From: Andreas Hansson [mailto:andreas.hans...@arm.com]
> Sent: Monday, October 17, 2016 3:29 AM
> To: gem5 users mailing list <gem5-us...@gem5.org gem5-us...@gem5.org>>
> Cc: Potter, Brandon 
> <brandon.pot...@amd.com<mailto:brandon.pot...@amd.com
> >>
> Subject: Re: [gem5-users] scons build error for Garnet 2.0
>
> My bad, this is properly broken for ALPHA and tru64.
>
> In fact this was broken as part of:
>
> changeset:   11383:5ac090acd180
> user:Brandon Potter <brandon.pot...@amd.com brandon.pot...@amd.com>>
> date:Thu Mar 17 10:24:17 2016 -0700
> summary: syscall_emul: extend mmap system call to support file backed
> mmap
>
> It was later fixed for Linux, but not tru64 it appears.
>
> Brandon, could you get this resolved?
>
> Thanks,
>
> Andreas
>
> From: gem5-users <gem5-users-boun...@gem5.org gem5-users-boun...@gem5.org>> on behalf of Andreas Hansson < 
> andreas.hans...@arm.com<mailto:andreas.hans...@arm.com>>
> Reply-To: gem5 users mailing list <gem5-us...@gem5.org gem5-us...@gem5.org>>
> Date: Monday, 17 October 2016 at 08:54
> To: gem5 users mailing list <gem5-us...@gem5.org gem5-us...@gem5.org>>
> Subject: Re: [gem5-users] scons build error for Garnet 2.0
>
> Just use clang (i.e. the default OSX compiler).
>
> Andreas
>
> From: gem5-users <gem5-users-boun...@gem5.org gem5-users-boun...@gem5.org>> on behalf of "F. A. Faisal" < 
> dipu.7...@gmail.com<mailto:dipu.7...@gmail.com>>
> Reply-To: gem5 users mailing list <gem5-us...@gem5.org gem5-us...@gem5.org>>
> Date: Saturday, 15 October 2016 at 14:22
> To: gem5 users mailing list <gem5-us...@gem5.org gem5-us...@gem5.org>>
> Subject: Re: [gem5-users] scons build error for Garnet 2.0
>
>
> I tried with another gcc, but the result is the same...
>
>
>
> $ port select --set gccmp-gcc49
>
>
>
> $ port select --list gcc
>
> Warning: port definitions are more than two weeks old, consider 
> updating them by running 'port selfupd

Re: [gem5-dev] macro (and identifier) problems in our source code

[Potter, Brandon]

Hi Andreas,

I thought that the _name designations were meant for private members with 
accessors; that's what the style guide says: "Class members that have accessor 
methods should have a leading underscore to indicate that the user should be 
using an accessor. The accessor functions themselves should have the same name 
as the variable without the leading underscore." It's not always an issue with 
the members being named with an underscore, but I have seen cases with acronyms 
as the first few characters with full capitalization... so an error according 
to the C++ standard.

The macros in "src/base/misc.hh" that are used extensively are named panic, 
warn, and fatal. These are examples which are particularly likely to cause 
conflicts (as I found out this past weekend). The problem manifested as an 
error after a macro called another macro caused by a conflict on a function 
name. So, preprocessor sees the "warn" define from "misc.hh" and then sees 
"warn" from . It has already seen the macro so it decides to start 
replacing the declaration in the  header file.  Since the "warn" macro 
from "misc.hh" called another macro it was tough to track the problem down; the 
compiler pointed me to the nested macro and I stared at that code and the 
surrounding code for a while before realizing the problem with . Also, 
the header that included "misc.hh" was nested in the header file a few includes 
away.  Frankly, it was a waste of time that other developers probably would 
like to avoid going forward. I'm suggesting added the extra characters so that 
no one else runs into the problem again (at least as long as we don't include 
header files from the OTHER gem5 project). I know that it will offset the code 
and likely cause some line issues that need to be resolved, but it seems like a 
better programming practice to adopt rather than maintain the status quo and 
continue to add code.  I do agree that it's unfortunate though to need the 
extra characters However, imagine how nice it would be now if it had been 
adopted earlier in development.  You'd be able to look at a macro and know that 
it's a macro without needing to resort to header files to verify; also, you'd 
know that it was unique and need not worry about collisions.

Regards,
Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Hansson
Sent: Monday, October 10, 2016 4:14 PM
To: gem5 Developer List <gem5-dev@gem5.org>
Subject: Re: [gem5-dev] macro (and identifier) problems in our source code

Hi Brandon,

No objection on the header include guards.

When it comes to private members, I would suggest to just stick with nameOfVar. 
We have only used _name when there is a constructor parameter name, and it is 
actually not necessary. It is fine to have the private member be just name, and 
let the constructor parameter name initialise it with name(name).

What macros are causing problems? Adding 5 characters to each and everyone 
seems rather unfortunate.

Andreas

On 10/10/2016, 18:52, "gem5-dev on behalf of Potter, Brandon"
<gem5-dev-boun...@gem5.org on behalf of brandon.pot...@amd.com> wrote:

>Hello all,
>
>I just posted a patch, http://reviews.gem5.org/r/3650/, which deals 
>with a name collision between a macro that's defined in our source and 
>a function declaration included from a standard header. Seems like a 
>rare collision that I stumbled into, but we should do a better job of 
>naming our macros given that collisions are a possibility. With macros, 
>we can't use namespaces to prevent this from happening, but we could 
>make a rule where we append something like "_GEM5" onto all of the 
>macros. (It is a much larger problem than function names which is what 
>the issue is on
>RB.) Does anyone have thoughts on this suggestion? If it's a good 
>suggestion, does anyone want to try their hand at sed magic? I don't 
>expect this is solvable with a 1-liner, but I don't want to spend time 
>trying to rename all of our macros. (There are ~300 of them in the 
>source right now that are function macros; there are a lot of defines.)
>
>In a related vein, our macro naming should never include __ (double
>underscore) anywhere in the macro nor should the macro begin with an 
>_[A-Z] (underscore followed by uppercase letter). It turns out that our 
>headers all use __X_Y_HH__ format for our include guards and I've found 
>a couple of example with variables that use the _[A-Z] format.  (It's 
>not just macros; the rule also applies to any identifier.) We need  to 
>fix these issues.  I do not know that it will create a problem given 
>that we're using the 'HH' in the macro, but it's bad form to continue 
>on with this. The standard dictates that these forms are reserved for 
>the implementation of the language 

[gem5-dev] macro (and identifier) problems in our source code

[Potter, Brandon]

Hello all,

I just posted a patch, http://reviews.gem5.org/r/3650/, which deals with a name 
collision between a macro that's defined in our source and a function 
declaration included from a standard header. Seems like a rare collision that I 
stumbled into, but we should do a better job of naming our macros given that 
collisions are a possibility. With macros, we can't use namespaces to prevent 
this from happening, but we could make a rule where we append something like 
"_GEM5" onto all of the macros. (It is a much larger problem than function 
names which is what the issue is on RB.) Does anyone have thoughts on this 
suggestion? If it's a good suggestion, does anyone want to try their hand at 
sed magic? I don't expect this is solvable with a 1-liner, but I don't want to 
spend time trying to rename all of our macros. (There are ~300 of them in the 
source right now that are function macros; there are a lot of defines.)

In a related vein, our macro naming should never include __ (double underscore) 
anywhere in the macro nor should the macro begin with an _[A-Z] (underscore 
followed by uppercase letter). It turns out that our headers all use __X_Y_HH__ 
format for our include guards and I've found a couple of example with variables 
that use the _[A-Z] format.  (It's not just macros; the rule also applies to 
any identifier.) We need  to fix these issues.  I do not know that it will 
create a problem given that we're using the 'HH' in the macro, but it's bad 
form to continue on with this. The standard dictates that these forms are 
reserved for the implementation of the language so we shouldn't be using them.

I recommend the following (although I am not volunteering to do it all):

*Change header guards to have the form: NAME_OF_FILE_HH_

*Use m_ (like Ruby) for private members in classes: m_nameOfVar as 
opposed to _nameOfVar; (consider something like _PID where the capitalization 
seems natural)

*Append a namespace substitute like "_GEM5" onto the end of macros (or 
alternatively use GEM5_ at the beginning)

Regards,
Brandon
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] Architecture of gem5

[Potter, Brandon]

Yeah, I remember the "Linux process trackers"; they were my motivation for 
exploring the memory reads on the kernel structures. I wanted to extend them, 
but the source was closed so I started hacking around to implement my own.

Unfortunately, I don't get to use FS here at AMD much; we rely almost 
exclusively on SE for now. So, I don't know if it's possible to make Python 
scripts interact with a running kernel in FS. I suspect that it might be 
possible to build the mechanism easily enough though. You could create Python 
wrappers for C++ methods that read physical memory; the memory should be 
accessible through the "System" class. To make the equivalent of HAPs, you 
could create pseudo instructions and instrument a Linux kernel with them. 
Alternatively, you could add hooks into gem5 on register state changes to do 
some magical thing that you want to do. Hopefully someone else has something 
more constructive to say, but that's what comes to mind for me.

The caveat with this kind of thing is that we would have to maintain it as the 
Linux kernel moves forward to keep it working. The kernel developers have a 
tendency to change the symbols that are made available through 
kallsyms/System.map. Currently, I think that they're not even visible by 
default with a kernel; they turned them off for security reasons. We'd have to 
compile a new kernel with the switch flipped on. Anyways, I might not be a big 
deal, because we already package our FS kernels and post them on the gem5 
website.

This is a project that I've wanted to work on for the past few years, but I 
haven't had the time to devote to it. However, I do agree that this would be an 
excellent feature to add for research purposes.
 
-Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Boris 
Shingarov/Employee/LW-US
Sent: Friday, September 9, 2016 8:03 AM
To: gem5 Developer List <gem5-dev@gem5.org>
Subject: Re: [gem5-dev] Architecture of gem5

I remember back when I was using Simics, there were these things called "Linux 
process trackers" that did exactly that. It  was all written in Python, and 
there was a point when various  processor-specific trackers got simplified / 
pulled together into one  generic tracker in which only the truly ISA-dependent 
knowledge was  pushed down to subclasses like x86_ops/ppc_ops etc., which would 
 override methods like "syscall_num()" according with passing that num in  eax 
vs r0; or overriding "current_task()" according to the struct being  kept on 
the kernel stack vs pointed by sprg3, etc.
It would be  extremely interesting to have similar functionality in gem5, and I 
have  even thought about writing something like that myself, but I don't know  
much about the infrastructure that would allow one to hook up such  Python 
scripts to haps like "memory access breakpoint" or "interrupt" or  "user/kernel 
mode change" easily and even on-the-fly in the middle of  debugging the kernel. 
 If someone could provide an initial explanation  of what mechanisms are there 
beyond using Python for initial  configuration, it would serve as a push 
forward at least for me.

-"gem5-dev" <gem5-dev-boun...@gem5.org> wrote: -
To: gem5 Developer List <gem5-dev@gem5.org>
From: "Potter, Brandon" 
Sent by: "gem5-dev" 
Date: 09/08/2016 07:26PM
Subject: Re: [gem5-dev] Architecture of gem5

With full-system mode, you can do kernel introspection. The kernel resides in 
the simulated memory which you have complete control over; you can do whatever 
you want to do with it. The kernel is what is responsible for managing the 
processes and their memory so if you want interesting kernel-level information, 
you might look into reading the kernel's physical memory directly.

With complete access to the full memory layout, you can play tricks with the 
physical memory and OS symbols to access structures, namely task_struct and 
mm_struct. It's difficult to do this and requires a bit of trial and error to 
figure out which symbols the kernel exposes to allow you to do this, but it is 
possible. The trick is to read the hex values from physical memory and compare 
them with the Linux kernel source; you can reason about which fields are 
pointers and which have values that are defined in the source. It just takes 
work to puzzle out the rest. I did this once with Simics so it is possible, but 
it's time consuming.

If you obtain access to the task_struct list, you can get access to the 
mm_struct which give you the full memory layout for each process. With the full 
memory layout, you can discern which areas of each process' virtual address 
space are mapped and which physical frames (in the simulated memory) correspond 
to those virtual address. In this way, you can get both the virtual and 
physical addresses of addresses for any process.

As J

Re: [gem5-dev] Architecture of gem5

[Potter, Brandon]

gem5/src/arch/x86/process.hh
gem5/src/arch/x86/process.cc
gem5/src/arch/x86/regs/misc.hh
gem5/src/arch/x86/regs/segment.hh

I'd look through full-system code and either find a hook or make one to allow 
access to the segment registers in the preceding files. It might take some time 
to figure that out, but this should get you started. 

-Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Jasmin Jahic
Sent: Friday, September 9, 2016 12:45 AM
To: gem5 Developer List <gem5-dev@gem5.org>
Subject: Re: [gem5-dev] Architecture of gem5

@Jason, @Brandon, thank you very much for your explanations and references.
I will try to solve this and if I succeed I will post the solution.

I just have one more technical question. Where in gem5 I can track registers of 
the CPU? I am able to identify fetch, decode, execute phases, as well as the 
access to the abstract memory and cache pages. I am using atomic/simple CPU.

Best regards,
Jasmin

On Fri, Sep 9, 2016 at 1:50 AM, Potter, Brandon <brandon.pot...@amd.com>
wrote:

> "If the sign extended bits are 1s, then the accesses are going 
> into kernel space and you can be reasonably sure that the kernel is 
> executing. So, the fool proof method is the CPL level in the code 
> segment register, but you can sometimes tell by the accesses if the 
> kernel is running or not."
>
> I just want to reiterate that this is not the best method, the kernel 
> can access user space virtual address so the kernel might be running 
> and you might see accesses to user-space virtual addresses, but it's 
> not guaranteed to tell whether you're in kernel mode or user mode. You 
> should really check the ring mode (if using X86).
>
> "With complete access to the full memory layout, you can play 
> tricks with the physical memory and OS symbols to access structures, 
> namely task_struct and mm_struct. It's difficult to do this and 
> requires a bit of trial and error to figure out which symbols the 
> kernel exposes to allow you to do this, but it is possible. The trick 
> is to read the hex values from physical memory and compare them with 
> the Linux kernel source; you can reason about which fields are 
> pointers and which have values that are defined in the source. It just 
> takes work to puzzle out the rest. I did this once with Simics so it is 
> possible, but it's time consuming."
>
> If you want to go down this rabbit hole, here are a couple of notes on 
> this. Firstly, you need to know about the kernel symbols (kallsyms), 
> https://onebitbug.me/2011/03/04/introducing-linux-kernel-symbols/. The 
> kallsyms get loaded from something in /boot (System.map I think or 
> some configuration file). Given these addresses, you can figure out 
> where the symbols are in physical memory and read their hex values.
>
> -Brandon
>
> -Original Message-
> From: Potter, Brandon
> Sent: Thursday, September 8, 2016 6:26 PM
> To: gem5 Developer List <gem5-dev@gem5.org>
> Subject: RE: [gem5-dev] Architecture of gem5
>
> With full-system mode, you can do kernel introspection. The kernel 
> resides in the simulated memory which you have complete control over; 
> you can do whatever you want to do with it. The kernel is what is 
> responsible for managing the processes and their memory so if you want 
> interesting kernel-level information, you might look into reading the 
> kernel's physical memory directly.
>
> With complete access to the full memory layout, you can play tricks 
> with the physical memory and OS symbols to access structures, namely 
> task_struct and mm_struct. It's difficult to do this and requires a 
> bit of trial and error to figure out which symbols the kernel exposes 
> to allow you to do this, but it is possible. The trick is to read the 
> hex values from physical memory and compare them with the Linux kernel 
> source; you can reason about which fields are pointers and which have 
> values that are defined in the source. It just takes work to puzzle 
> out the rest. I did this once with Simics so it is possible, but it's time 
> consuming.
>
> If you obtain access to the task_struct list, you can get access to 
> the mm_struct which give you the full memory layout for each process. 
> With the full memory layout, you can discern which areas of each 
> process' virtual address space are mapped and which physical frames 
> (in the simulated
> memory) correspond to those virtual address. In this way, you can get 
> both the virtual and physical addresses of addresses for any process.
>
> As Jason mentioned, you can use /proc/{some_pid}/maps to figure out 
> the virtual address ranges that are mapped (at a specific point in the 
> execution of that process). Furthermo

Re: [gem5-dev] Architecture of gem5

[Potter, Brandon]

"If the sign extended bits are 1s, then the accesses are going into 
kernel space and you can be reasonably sure that the kernel is executing. So, 
the fool proof method is the CPL level in the code segment register, but you 
can sometimes tell by the accesses if the kernel is running or not."

I just want to reiterate that this is not the best method, the kernel can 
access user space virtual address so the kernel might be running and you might 
see accesses to user-space virtual addresses, but it's not guaranteed to tell 
whether you're in kernel mode or user mode. You should really check the ring 
mode (if using X86).

"With complete access to the full memory layout, you can play tricks 
with the physical memory and OS symbols to access structures, namely 
task_struct and mm_struct. It's difficult to do this and requires a bit of 
trial and error to figure out which symbols the kernel exposes to allow you to 
do this, but it is possible. The trick is to read the hex values from physical 
memory and compare them with the Linux kernel source; you can reason about 
which fields are pointers and which have values that are defined in the source. 
It just takes work to puzzle out the rest. I did this once with Simics so it is 
possible, but it's time consuming."

If you want to go down this rabbit hole, here are a couple of notes on this. 
Firstly, you need to know about the kernel symbols (kallsyms), 
https://onebitbug.me/2011/03/04/introducing-linux-kernel-symbols/. The kallsyms 
get loaded from something in /boot (System.map I think or some configuration 
file). Given these addresses, you can figure out where the symbols are in 
physical memory and read their hex values.

-Brandon

-Original Message-
From: Potter, Brandon 
Sent: Thursday, September 8, 2016 6:26 PM
To: gem5 Developer List <gem5-dev@gem5.org>
Subject: RE: [gem5-dev] Architecture of gem5

With full-system mode, you can do kernel introspection. The kernel resides in 
the simulated memory which you have complete control over; you can do whatever 
you want to do with it. The kernel is what is responsible for managing the 
processes and their memory so if you want interesting kernel-level information, 
you might look into reading the kernel's physical memory directly.

With complete access to the full memory layout, you can play tricks with the 
physical memory and OS symbols to access structures, namely task_struct and 
mm_struct. It's difficult to do this and requires a bit of trial and error to 
figure out which symbols the kernel exposes to allow you to do this, but it is 
possible. The trick is to read the hex values from physical memory and compare 
them with the Linux kernel source; you can reason about which fields are 
pointers and which have values that are defined in the source. It just takes 
work to puzzle out the rest. I did this once with Simics so it is possible, but 
it's time consuming.

If you obtain access to the task_struct list, you can get access to the 
mm_struct which give you the full memory layout for each process. With the full 
memory layout, you can discern which areas of each process' virtual address 
space are mapped and which physical frames (in the simulated memory) correspond 
to those virtual address. In this way, you can get both the virtual and 
physical addresses of addresses for any process.

As Jason mentioned, you can use /proc/{some_pid}/maps to figure out the virtual 
address ranges that are mapped (at a specific point in the execution of that 
process). Furthermore, you can use pagemap to figure out what the physical 
address is. The catch with using these methods is that the simulator has to be 
running and the process that you're interested in examining has to be running 
to read the files (because the maps and pagemaps features are pseudo files). If 
you have a short-running process, you typically have to add a `while(1)` loop 
into the application to figure out the mappings.

Also as Jason mentioned on X86, you can examine the ring level that the CPU is 
in to figure out if the kernel is executing or if the process is running in 
userspace; if you don't understand what we're talking about, read 
http://duartes.org/gustavo/blog/post/cpu-rings-privilege-and-protection/. You 
can examine the segmentation registers and figure out at any given time what's 
going on. (If you don't know much about memory, I'd read the rest of his blog 
posts on the topic as well. It's a nice summary of Linux memory for the 
uninitiated.) Also, userspace processes should not have access to kernel space 
data. Linux draws a line in the virtual memory sand for different architectures 
to specify what is "kernel space" and what is "user space". With X86-64, 48 
bits [47...0] are used for virtual addresses. (The 47th bit gets sign-extended 
to the 63rd bit so that all of the intervening bits don't really matter.) If 
the sign extended bits are 1s, then the

Re: [gem5-dev] Architecture of gem5

[Potter, Brandon]

With full-system mode, you can do kernel introspection. The kernel resides in 
the simulated memory which you have complete control over; you can do whatever 
you want to do with it. The kernel is what is responsible for managing the 
processes and their memory so if you want interesting kernel-level information, 
you might look into reading the kernel's physical memory directly.

With complete access to the full memory layout, you can play tricks with the 
physical memory and OS symbols to access structures, namely task_struct and 
mm_struct. It's difficult to do this and requires a bit of trial and error to 
figure out which symbols the kernel exposes to allow you to do this, but it is 
possible. The trick is to read the hex values from physical memory and compare 
them with the Linux kernel source; you can reason about which fields are 
pointers and which have values that are defined in the source. It just takes 
work to puzzle out the rest. I did this once with Simics so it is possible, but 
it's time consuming.

If you obtain access to the task_struct list, you can get access to the 
mm_struct which give you the full memory layout for each process. With the full 
memory layout, you can discern which areas of each process' virtual address 
space are mapped and which physical frames (in the simulated memory) correspond 
to those virtual address. In this way, you can get both the virtual and 
physical addresses of addresses for any process.

As Jason mentioned, you can use /proc/{some_pid}/maps to figure out the virtual 
address ranges that are mapped (at a specific point in the execution of that 
process). Furthermore, you can use pagemap to figure out what the physical 
address is. The catch with using these methods is that the simulator has to be 
running and the process that you're interested in examining has to be running 
to read the files (because the maps and pagemaps features are pseudo files). If 
you have a short-running process, you typically have to add a `while(1)` loop 
into the application to figure out the mappings.

Also as Jason mentioned on X86, you can examine the ring level that the CPU is 
in to figure out if the kernel is executing or if the process is running in 
userspace; if you don't understand what we're talking about, read 
http://duartes.org/gustavo/blog/post/cpu-rings-privilege-and-protection/. You 
can examine the segmentation registers and figure out at any given time what's 
going on. (If you don't know much about memory, I'd read the rest of his blog 
posts on the topic as well. It's a nice summary of Linux memory for the 
uninitiated.) Also, userspace processes should not have access to kernel space 
data. Linux draws a line in the virtual memory sand for different architectures 
to specify what is "kernel space" and what is "user space". With X86-64, 48 
bits [47...0] are used for virtual addresses. (The 47th bit gets sign-extended 
to the 63rd bit so that all of the intervening bits don't really matter.) If 
the sign extended bits are 1s, then the accesses are going into kernel space 
and you can be reasonably sure that the kernel is executing. So, the fool proof 
method is the CPL level in the code segment register, but you can sometimes 
tell by the accesses if the kernel is running or not. If you want to learn 
about memory in Linux, the canonical document, at least in my mind is: 
https://www.kernel.org/doc/gorman/. (Although, the book is a bit dated.)

If you're looking to track a specific process in X86, you should be able to 
monitor the CR3 register to verify that the correct process is running; the CR3 
holds the base frame for the page tables so it's a decent way to figure out if 
a 'special' process is being run.

-Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Jason Lowe-Power
Sent: Wednesday, August 31, 2016 8:59 AM
To: gem5 Developer List 
Subject: Re: [gem5-dev] Architecture of gem5

Hi Jasmin,

In full-system mode, gem5 runs a full operating system and all software.It is 
analogous to a pure virtualization platform. In full-system mode, gem5 must 
model all devices, etc. that the OS expects to interact with.
This is in comparison to syscall-emulation mode. In SE mode, gem5 only executes 
user-mode code. All OS system calls are routed into the simulator and are 
*emulated*.

To answer your questions:
   - Is it possible to reason wether an instruction is user instruction or 
kernel instruction?
Yes and no. No, there is no simple function to call to see if you are currently 
running in kernel or user mode. However, depending on your kernel / OS certain 
PC addresses represent kernel vs user-mode code. Additionally, you could watch 
what mode the CPU is in (ring-0 vs ring-3, etc), depending on the architecture.

   - Can we know to which process is an instruction belongs inside of the  OS?
This is a little more tricky, but it may be possible based on the physical 
address of the PC 

Re: [gem5-dev] X86 RSP return address (after MemWrite) not yet updated issue?

[Potter, Brandon]

Hi Bjoern,

Did you ever solve this issue?  I see what you're describing, but it's not 
obvious to me what causes the problem.

Thanks,
Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Bjoern A. Zeeb
Sent: Monday, August 15, 2016 8:08 AM
To: gem5 Developer List 
Subject: [gem5-dev] X86 RSP return address (after MemWrite) not yet updated 
issue?

Hi,

I was trying to skip FreeBSD’s DELAY() on X86_64 very much like we do on ARM 
for Linux (or FreeBSD for that matter) and started to implement things and 
found a strange behaviour:

 From my src/arch/x86/utility.cc

void
skipFunction(ThreadContext *tc)
{
 PCState newPC = tc->pcState();
 Addr sp = tc->readIntReg(INTREG_RSP); DPRINTF(XXXBZ, "XXX-BZ sp %#x\n", 
sp);
 Addr npc;
 // XXX For some reason the memory write is not visible yet *sigh*
 //CopyOut(tc, , sp, sizeof(Addr));
 FSTranslatingPortProxy  = tc->getVirtProxy();
 proxy.readBlob(sp, (uint8_t *), sizeof(Addr)); DPRINTF(XXXBZ, "XXX-BZ 
npc %#x\n", npc);
 newPC.set(npc);
 // Don't forget to increment the sp again.
 tc->setIntReg(INTREG_RSP, sp + 8);
 tc->pcState(newPC);
}


As you can see I tried two ways to read the return address off the stack, and 
neither (on the first try) returns the current one (after the memory write) but 
the previous one, which makes the preceding function part since the last ret to 
be run twice and on the 2nd iteration the memory location on the stack returns 
the proper (former) return address 
and we continue.   I would expected the correct value to be visible 
given the instruction was committed and logged with the DPRINTF.

That’s not the behaviour I expected.  Is there anything I am doing wrong or is 
this a (caching) bug?  Can anyone enlighten me?

My command line (including private options):

command line: ./build/X86/gem5.opt -r -e -d m5out-amd64-1 
--stdout-file=fbsd301452-detailed-00117.log
--stderr-file=fbsd301452-detailed-00117.err --debug-flags=Exec,XXXBZ 
configs/example/fs.py --mem-size=1024MB --os-type=FreeBSD --virtblk 
--loader-config-file=loader-amd64.conf --cpu-type=detailed 
--disk-image=disk-amd64-r301452.img --kernel=kernel-amd64-r301452 
--command-line=-hvs --caches --l2cache --l3cache --simple-trace-en

Bjoern



222604924000: system.cpu T0 : @_vprintf+255: ret
222604924000: system.cpu T0 : @_vprintf+255.0  :   RET_NEAR : ld   t1, 
SS:[rsp] : MemRead :  D=0x803e4d23 A=0x80974b98
222604924500: system.cpu T0 : @_vprintf+255.1  :   RET_NEAR : addi   
rsp, rsp, 0x8 : IntAlu :  D=0x80974ba0
222604924500: system.cpu T0 : @_vprintf+255.2  :   RET_NEAR : wripi   , 
t1, 0 : IntAlu :
222604933000: system.cpu T0 : @printf+83: cmp   
DS:[0x8095c638], 0
222604933000: system.cpu T0 : @printf+83.0  :   CMP_M_I : limm   t2, 0   
: IntAlu :  D=0x
222604933000: system.cpu T0 : @printf+83.1  :   CMP_M_I : ld   t1, 
DS:[0x8095c638] : MemRead :  D=0x
A=0x8095c638
222604933000: system.cpu T0 : @printf+83.2  :   CMP_M_I : sub   t0, t1, 
t2 : IntAlu :  D=0x
222604933000: system.cpu T0 : @printf+92: jnz   0xb
222604933000: system.cpu T0 : @printf+92.0  :   JNZ_I : rdip   t1, 
%ctrl153,  : IntAlu :  D=0x803e4d2e
222604933000: system.cpu T0 : @printf+92.1  :   JNZ_I : limm   t2, 0xb   
: IntAlu :  D=0x000b
222604933000: system.cpu T0 : @printf+92.2  :   JNZ_I : wrip   , t1, t2  
: IntAlu :
222604933000: system.cpu T0 : @printf+94: mov   
DS:[0x8095cab8], 0x1
222604933000: system.cpu T0 : @printf+94.0  :   MOV_M_I : limm   t1d, 
0x1 : IntAlu :  D=0x0001
222604933000: system.cpu T0 : @printf+94.1  :   MOV_M_I : st   t1d, 
DS:[0x8095cab8] : MemWrite :  D=0x0001
A=0x8095cab8
222604933500: system.cpu T0 : @printf+105: add  rax, 0x50
222604933500: system.cpu T0 : @printf+105.0  :   ADD_R_I : limm   t1, 
0x50 : IntAlu :  D=0x0050
222604933500: system.cpu T0 : @printf+105.1  :   ADD_R_I : add   rsp, 
rsp, t1 : IntAlu :  D=0x
222604933500: system.cpu T0 : @printf+109: pop  rbp
222604933500: system.cpu T0 : @printf+109.0  :   POP_R : ld   t1, 
SS:[rsp] : MemRead :  D=0x80974c60 A=0x80974bf0
222604933500: system.cpu T0 : @printf+109.1  :   POP_R : addi   rsp, 
rsp, 0x8 : IntAlu :  D=0x80974bf8
222604933500: system.cpu T0 : @printf+109.2  :   POP_R : mov   rbp, rbp, 
t1 : IntAlu :  D=0x80974c60
222604933500: system.cpu T0 : @printf+110: ret
222604933500: system.cpu T0 : @printf+110.0  :   RET_NEAR : ld   t1, 
SS:[rsp] : MemRead :  D=0x80611d2e A=0x80974bf8
222604933500: system.cpu T0 : @printf+110.1  :   RET_NEAR : addi   rsp, 
rsp, 0x8 : IntAlu :  D=0x80974c00
222604933500: system.cpu T0 : @printf+110.2  :   RET_NEAR : wripi   , 
t1, 0 : IntAlu :
222604944000: system.cpu T0 : 

Re: [gem5-dev] Question about newfstatat() syscall

[Potter, Brandon]

Hello Sooraj,

This changeset from the Linux git repo looks relevant:
  5590ff0d5528b60153c0b4e7b771472b5a95e297

It dates back to 01/18/2006 when Drepper added some system calls to the Linux 
source.

  $> git diff 5590ff0d5528b60153c0b4e7b771472b5a95e297^ 
5590ff0d5528b60153c0b4e7b771472b5a95e297
  diff --git a/fs/stat.c b/fs/stat.c
  index b8a0e51..24211b0 100644
  --- a/fs/stat.c
  +++ b/fs/stat.c

  +asmlinkage long sys_newfstatat(int dfd, char __user *filename,
  +   struct stat __user *statbuf, int flag)
  +{
  +   struct kstat stat;
  +   int error = -EINVAL;
  +
  +   if ((flag & ~AT_SYMLINK_NOFOLLOW) != 0)
  +   goto out;
  +
  +   if (flag & AT_SYMLINK_NOFOLLOW)
  +   error = vfs_lstat_fd(dfd, filename, );
  +   else
  +   error = vfs_stat_fd(dfd, filename, );
  +
  +   if (!error)
  +   error = cp_new_stat(, statbuf);
  +
  +out:
  +   return error;
  +}

Unfortunately, I cannot find an explanation as to why these calls were added. 
From what I see here, I'm guessing that new functionality was originally needed 
to allow fstatat to follow (or not follow) symlinks.

Looking at the tip of the Linux repository, I can see that more functionality 
was built on top of the system call. For instance, AT_EMPTY_PATH, 
AT_NO_AUTOMOUNT, and AT_SYMLINK_NOFOLLOW are all options now (which seems to 
agree with man pages). There also appears to be an issue with architectures 
depending on if they're 64-bit or 32-bit; some architectures prefer fstatat64 
instead of newfstatat. So, it looks like the kernel chooses a buffer size 
depending on which system call is invoked. If gem5 calls the wrong host system 
call internally (with the GLIBC wrappers), it will likely clobber the simulated 
memory and cause silent memory corruption. The key is to ensure that the host, 
simulator, and application all are expecting to use the same buffer size or 
you'll need to code around it to accommodate.

The best thing to do is to probably pull down the Linux source and decide 
what's best for yourself; the system call implementations for these functions 
still reside in "linux/fs/stat.c". Essentially, you need to figure out how the 
options are really being handled internally and then emulate that behavior in 
gem5; you'll also need to understand the contract between GLIBC and the kernel. 
There usually is not too much documentation (in the kernel or GLIBC) so I've 
had to resort to searching through source before to figure it out.

Regards,
Brandon

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Sooraj Puthoor
Sent: Wednesday, April 20, 2016 5:27 PM
To: gem5-dev@gem5.org
Subject: [gem5-dev] Question about newfstatat() syscall

Hi all,

I am trying to run a benchmark in gem5 and while running the benchmark, I am 
getting “fatal: syscall newfstatat (#262) unimplemented” error.

It seems gem5 has a "fstatat()" syscall defined but no definition for "
newfstatat()" exists. So, I was thinking of redirecting the "newstatat()"
syscall to the already defined "fstatat()" syscall. I am not sure if this the 
correct approach and would like to know the thoughts of folks who might have 
already experienced same/similar problem.

Another option is to try to remove the syscall from the benchmark by modifying 
the code and/or recompiling the code. But this syscall comes from a 
pre-compiled library that I am using and I would go down the path of 
recompiling that library only as my last option :)

Thank you
-Sooraj Puthoor
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] Reviews: dynamic linkage support for SE mode

[Potter, Brandon]

Hello developers,

I recently posted patches to reviewboard to add support for dynamic linking and 
loading in se mode.  I still do not have reviews for the loader patches:
http://reviews.gem5.org/r/3327/
http://reviews.gem5.org/r/3328/
http://reviews.gem5.org/r/3329/

Planning on pushing these on March 4th unless someone posts reviews before that 
date.

Thanks,
Brandon
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] Fwd: Line Address in SLICC

[Potter, Brandon]

Hi,

We should be able to figure it out with a little digging.

It appears that the output is generated with "--debug-flags=RubySlicc".  From 
the "*.sm" files, we can see that some of the actions have:
DPRINTF(RubySlicc, "Address: %#x, Data Block: %s\n", address, tbe.DataBlk);

These "*.sm" files are used to help generate code when gem5 compiles the Ruby 
protocol (which it seems that we're generating MOESI_CMP_directory).  Within 
the build folder in gem5, execute:
find ./build | xargs grep -I "$ACTION_NAME"
where $ACTION_NAME is the action name that encapsulates the DPRINTF above.

With "c_sendData..." supplied, it should generate a list of files like the 
following:
./build/X86/mem/protocol/L2Cache_Transitions.cc:
c_sendDataFromTBEToL1GETS(m_tbe_ptr, m_cache_entry_ptr, addr);
./build/X86/mem/protocol/L2Cache_Controller.hh:void 
c_sendDataFromTBEToL1GETS(L2Cache_TBE*& m_tbe_ptr, L2Cache_Entry*& ...

./build/X86/mem/protocol/L2Cache_Controller.cc:L2Cache_Controller::c_sendDataFromTBEToL1GETS(L2Cache_TBE*&
 m_tbe_ptr, ...
./build/X86/mem/protocol/L2Cache_Controller.cc:DPRINTF(RubyGenerated, 
"executing c_sendDataFromTBEToL1GETS\n");
./build/X86/mem/protocol/L2Cache_Controller.cc:   fatal("Error in 
action L2Cache:c_sendDataFromTBEToL1GETS: "

We figure out where the declaration is in the files above (looks like it's in 
"L2CacheController.cc").

In examining the file, we notice that "addr" is passed as a parameter into the 
functions.  The only place that it looks to be called from is 
"L2Cache_Transitions.cc"; it appears under "doTransitionWorker" which is called 
from "doTransition".
find ./build | xargs grep -I "doTransition"

This generates many files, but L2Cache_Wakeup.cc looks promising.  From 
examining the file, it looks like all of the addresses come from 
"((in_msg_ptr)).m_addr".  Seems like "in_msg_ptr" can any one of multiple 
message types (RequestMsg, TriggerMsg, ResponseMsg, etc..).
find ./build -name "RequestMsg.*"

Within the resulting files (Request.hh, Request.cc), it's not explicitly clear 
if the address is a line address or a normal address.  (It mentions that the 
address is a physical address though.)  However, we notice the 
"RequestMsg::print" statement.  Specifically, look at printAddress(m_addr).
find ./build | xargs grep -I "printAddress".

The definition of "printAddress" comes from 
"build/***/mem/ruby/common/Address.cc".  The function is masking the low order 
bits:
"<< maskLowOrderBits(addr, RubySystem::getBlockSizeBits())"
Given this information, it only makes sense for the m_addr in the message files 
to be a full address and not the line address.

Also, the data block output has the form: [ 0xXX 0xXX]; a byte can also be 
represented as 0xXX.  There are 64 of these 0xXX (just like a cache line).  So, 
it makes sense to suspect that this is the data for the line address that our 
full address would create.  You might try to verify that. :)

Regards,
Brandon

From: gem5-dev  on behalf of P Pinky 

Sent: Thursday, October 29, 2015 5:13 AM
To: gem5 Developer List
Subject: [gem5-dev] Fwd: Line Address in SLICC

Hi all

 I have tried to print the line address in protocol files from the
in_msg.LineAddress.

 So by tracing the file, i got the following output.

355500: system.ruby.l1_cntrl0: MOESI_CMP_directory-L1cache.sm:364: *Line
address: 2097216*

407000: system.ruby.l2_cntrl0: MOESI_CMP_directory-L2cache.sm:799: *Address:
2097216*, Data Block: [ 0x1 0x2d 0xca 0x17 0x0 0x0 0x48 0x1 0x2d 0xab 0x1f
0x0 0x0 0x48 0x1 0x2d 0xac 0x1f 0x0 0x0 0x48 0x1 0x2d 0x95 0x3f 0x0 0x0
0x48 0x8d 0x3d 0x9e 0xff 0xff 0xff 0x48 0x81 0xe7 0x0 0x0 0xe0 0xff 0x48
0x89 0xf8 0x48 0xc1 0xe8 0x1e 0x48 0x25 0xff 0x1 0x0 0x0 0x74 0x33 0x48
0x8d 0x95 0x63 0x60 0x20 0x0 0x48 ]

the one in red color is what i printed using in_msg.LineAddress in L1
cache.sm

the one in green was in L2cache.sm but its address (not specified as line
address).

So am confused now whether it "2097216" is line address or not. Can some
one clarify this? Also 0x1,ox2d  refers to datablocks. Can someone help me
with this?

I tried to change the L1cache size and associativity.But still the same
address is printing.So this means its not line address ? Then why
in_msg.LineAddress ?

THanks in advance
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] Commit Reviews 2966-2969 07/21/15

[Potter, Brandon]

Hello all,

I would like to commit the following reviews (related to process class and SE 
mode):
http://reviews.m5sim.org/r/2966/
http://reviews.m5sim.org/r/2967/
http://reviews.m5sim.org/r/2968/
http://reviews.m5sim.org/r/2969/

If anyone would like more time to review them, please speak up now.  Otherwise, 
I plan to turn them into changesets tomorrow.

Thanks,
Brandon
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] Requests Without Reviews

[Potter, Brandon]

Hello all,

I have two requests sitting in ReviewBoard that do not have reviews yet.  The 
requests were posted two months ago.
http://reviews.m5sim.org/r/2640/
http://reviews.m5sim.org/r/2641/

I would like to push these patches if no one has objections.  I plan to do so 
this Friday (04/17/15).

Thanks,
Brandon
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] changeset in gem5: ruby: allow restoring from checkpoint when us...

[Potter, Brandon]

Hello Joel,

Not looking to pick sides here, but I have also encountered problems with 
global/static variables in Ruby as well.  This past December, I spent a couple 
of weeks trying to clean up the Ruby code so that we could invoke two instances 
of Ruby at the same time in the simulator.  It was not trivial as the problems 
extended into slicc auto-generated code as well and I ended up temporarily 
abandoning it.  The high level code organization and design decisions should 
not prevent us from doing simple straight-forward things like invoking two Ruby 
instances at the same time.

When I was trying to solve the problem, I independently followed the same 
method as Nilay.  I added ruby system pointers to most of the objects so that 
they could back-reference the system object (since it's the central point of 
that Ruby instance) and obtain access to the other subsidiary objects.  I felt 
really dirty while writing the code and frankly the code was painful to look 
at, but there didn't appear to be a better method (at least not one that I 
could see).  A couple of folks that I spoke with here at AMD felt that the 
global/static variables needed to be fixed, but agreed that there was no easy 
solution.

Regards,
Brandon

From: gem5-dev [gem5-dev-boun...@gem5.org] on behalf of Nilay Vaish 
[ni...@cs.wisc.edu]
Sent: Tuesday, April 14, 2015 11:33 AM
To: gem5 Developer List
Subject: Re: [gem5-dev] changeset in gem5: ruby: allow restoring from 
checkpoint when us...

On Tue, 14 Apr 2015, Joel Hestness wrote:

 This patch didn't address my requested change (
 http://reviews.gem5.org/r/2702/), and shouldn't have been checked in. The
 AbstractController should not have or even need another pointer to the
 RubySystem, because it can access it with g_system_ptr. This is a dead
 simple fix. Adding the pointer is going in the wrong direction for
 abstraction.


Joel, in my opinion we should be eliminating global / static variables.
I have personally found them to be a problem while trying to parallelize
Ruby.

--
Nilay
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev