Re: [gem5-dev] Status of RISC-V patches

[Gutierrez, Anthony]

Hi Andreas,

I agree with this sentiment overall as I think our collection of regressions 
currently are pretty ad-hoc; they are basically just the benchmarks that people 
want to ensure run for their in gem5, i.e, SPEC etc. I think smaller LLVM unit 
tests would provide more directed testing, similar or better coverage, less 
maintenance, and a suitable license.

Just to clarify when you say 'LLVM-specific' tests, are you referring to the 
unit/regression tests: http://llvm.org/docs/TestingGuide.html#regression-tests 
? We are also thinking of using similar HCC tests (AMD's LLVM-based compiler 
for heterogeneous accelerators): 
https://github.com/RadeonOpenCompute/hcc/tree/clang_tot_upgrade/tests/Unit for 
our GPU model in gem5.

For testing, I'm assuming we're going to want something like Jenkins going 
forward. The question is where will such a server live, and will it be 
accessible to the public?

-Tony

-Original Message-
From: gem5-dev [mailto:gem5-dev-boun...@gem5.org] On Behalf Of Andreas Hansson
Sent: Monday, November 14, 2016 2:40 PM
To: gem5 Developer List <gem5-dev@gem5.org>; Jason Lowe-Power 
<ja...@lowepower.com>
Subject: Re: [gem5-dev] Status of RISC-V patches

Hi all,

Merely a thought when it comes to adding tests:

I would suggest we ditch all proprietary/closed-source tests and move in the 
direction of something that is open and maintained. My proposal would be to 
adopt a few of the tests from the llvm test suite. There are both very short 
LLVM-specific tests that are BSD licensed, and a bunch of smaller “apps” that 
maintain their original license. It would be great if we could identify a few 
relevant tests from the tests suite and start going in that direction. The 
source for the tests along with build scripts etc should probably be in a repo 
on their own.

What do you think?

Andreas

On 28/10/2016, 22:12, "gem5-dev on behalf of Alec Roelke"
<gem5-dev-boun...@gem5.org on behalf of ar...@virginia.edu> wrote:

>I think I get it.  Thanks for your help!
>
>It looks like I won't be able to make use of any tests other than 
>00.hello, because either they're multithreaded or pieces of them are 
>missing.  I'm going to try to put my instruction tests into 
>02.insttest.
>
>On Fri, Oct 28, 2016 at 11:01 AM, Jason Lowe-Power 
><ja...@lowepower.com>
>wrote:
>
>> Hi Alec,
>>
>> Our regression testing infrastructure is confusing, to say the least.
>> Honestly, it would take me at least a few hours to figure out how to 
>>add  new regressions again. But here's a little information that 
>>hopefully will  help you save some time.
>>
>> You need to put reference outputs in tests/quick/se/> NAME>/ref/riscv/linux/.
>>
>> The config files are in tests/configs. One of these files (without 
>>the
>> .py) is what goes in  above. The  can be  
>>anything. Using some of the same tests that are there make sense, but 
>>you  may want to add other RISC-V specific tests as well.
>>
>> You *may* be able to use the config files in tests/configs without  
>>changes, but maybe not. I'm not sure. You'll probably be able to use 
>>some  but not others.
>>
>> Getting scons to run your regressions is totally black magic to me.
>>Here's
>> a couple of things I've learned, though.
>> 1) You should delete the build directory (e.g., build/RISCV/tests/
>> debug/quick/se/00.hello/riscv/) every time you want to re-run the  
>>regression.
>> 2) The regression won't even try to run if you don't have dummy files 
>>in  tests/quick/se//ref/riscv/linux/.
>>
>> For the missing stats/changed stats, since this is first time you're  
>>running the RISC-V stats, I would totally ignore all of that 
>>information. I  would just look at the output (simout/simerr) and make 
>>sure it seems  reasonable (e.g., no errors). If you have an idea of a 
>>stat that you expect  to see (e.g., floating point ops > 0) then you 
>>could look at the stats file  too to make sure it's what you expect.
>>
>> There's no specific configuration you should be using. For the most 
>>part,  all of the tests are just functional tests. The configurations 
>>for the  system are specified in the tests/configs (>NAME>) Python files.
>>
>> Hopefully this will help you get started. Let us know if you have 
>> more questions. Maybe someone with more regression tester experience 
>> will be able to help more.
>>
>> Jason
>>
>> On Thu, Oct 27, 2016 at 8:23 PM Alec Roelke <ar...@virginia.edu> wrote:
>>
>>> I'll start with converting as many from "quick" as I can.  If/when I 
>>>end  up creating my own, is there a convention for what they should 
>>>be 

Re: [gem5-dev] Status of RISC-V patches

[Jason Lowe-Power]

Hi Andreas,

I don't think it's surprising, but I like this idea. I think it's much more
maintainable going forward than what we've been doing.

Does this have any bearing on the RISC-V patches? I was hoping we would be
able to get Alec's stuff pushed in soon. I don't want to hold things up for
perfection, especially since it seems a number of people have been asking
of RISC-V support.

Thanks,
Jason

On Mon, Nov 14, 2016 at 4:40 PM Andreas Hansson 
wrote:

> Hi all,
>
> Merely a thought when it comes to adding tests:
>
> I would suggest we ditch all proprietary/closed-source tests and move in
> the direction of something that is open and maintained. My proposal would
> be to adopt a few of the tests from the llvm test suite. There are both
> very short LLVM-specific tests that are BSD licensed, and a bunch of
> smaller “apps” that maintain their original license. It would be great if
> we could identify a few relevant tests from the tests suite and start
> going in that direction. The source for the tests along with build scripts
> etc should probably be in a repo on their own.
>
> What do you think?
>
> Andreas
>
> On 28/10/2016, 22:12, "gem5-dev on behalf of Alec Roelke"
>  wrote:
>
> >I think I get it.  Thanks for your help!
> >
> >It looks like I won't be able to make use of any tests other than
> >00.hello,
> >because either they're multithreaded or pieces of them are missing.  I'm
> >going to try to put my instruction tests into 02.insttest.
> >
> >On Fri, Oct 28, 2016 at 11:01 AM, Jason Lowe-Power 
> >wrote:
> >
> >> Hi Alec,
> >>
> >> Our regression testing infrastructure is confusing, to say the least.
> >> Honestly, it would take me at least a few hours to figure out how to add
> >> new regressions again. But here's a little information that hopefully
> >>will
> >> help you save some time.
> >>
> >> You need to put reference outputs in tests/quick/se/ >> NAME>/ref/riscv/linux/.
> >>
> >> The config files are in tests/configs. One of these files (without the
> >> .py) is what goes in  above. The  can be
> >> anything. Using some of the same tests that are there make sense, but
> >>you
> >> may want to add other RISC-V specific tests as well.
> >>
> >> You *may* be able to use the config files in tests/configs without
> >> changes, but maybe not. I'm not sure. You'll probably be able to use
> >>some
> >> but not others.
> >>
> >> Getting scons to run your regressions is totally black magic to me.
> >>Here's
> >> a couple of things I've learned, though.
> >> 1) You should delete the build directory (e.g., build/RISCV/tests/
> >> debug/quick/se/00.hello/riscv/) every time you want to re-run the
> >> regression.
> >> 2) The regression won't even try to run if you don't have dummy files in
> >> tests/quick/se//ref/riscv/linux/.
> >>
> >> For the missing stats/changed stats, since this is first time you're
> >> running the RISC-V stats, I would totally ignore all of that
> >>information. I
> >> would just look at the output (simout/simerr) and make sure it seems
> >> reasonable (e.g., no errors). If you have an idea of a stat that you
> >>expect
> >> to see (e.g., floating point ops > 0) then you could look at the stats
> >>file
> >> too to make sure it's what you expect.
> >>
> >> There's no specific configuration you should be using. For the most
> >>part,
> >> all of the tests are just functional tests. The configurations for the
> >> system are specified in the tests/configs () Python
> >>files.
> >>
> >> Hopefully this will help you get started. Let us know if you have more
> >> questions. Maybe someone with more regression tester experience will be
> >> able to help more.
> >>
> >> Jason
> >>
> >> On Thu, Oct 27, 2016 at 8:23 PM Alec Roelke  wrote:
> >>
> >>> I'll start with converting as many from "quick" as I can.  If/when I
> >>>end
> >>> up creating my own, is there a convention for what they should be
> >>>named and
> >>> where they should go?
> >>>
> >>> Also, I'm having a bit of a problem with just 00.hello.  I compiled the
> >>> source in tests/test-progs/hello/src into
> >>>tests/test-progs/hello/bin/riscv/linux
> >>> (for some reason mine disappeared), and then created
> >>> tests/quick/se/00.hello/ref/riscv/linux/simple-atomic.  Then I compiled
> >>> and ran build/RISCV/gem5.fast, configuring for the atomic CPU and
> >>> redirecting the output stats, configuration, stdout, and stderr into
> >>>that
> >>> directory.  But when I run build/RISCV/tests/debug/quick/
> >>> se/00.hello/riscv/linux/simple-atomic, it claims that it couldn't find
> >>> several stats and that it found several other unexpected ones in
> >>>stats.txt
> >>> (and completely ignored the other files).  Is there some other
> >>> configuration I should be using to generate the output for the
> >>>regression?
> >>> Actually, better yet, is there a way for me to figure out what the
> >>> 

Re: [gem5-dev] Status of RISC-V patches

[Andreas Hansson]

Hi all,

Merely a thought when it comes to adding tests:

I would suggest we ditch all proprietary/closed-source tests and move in
the direction of something that is open and maintained. My proposal would
be to adopt a few of the tests from the llvm test suite. There are both
very short LLVM-specific tests that are BSD licensed, and a bunch of
smaller “apps” that maintain their original license. It would be great if
we could identify a few relevant tests from the tests suite and start
going in that direction. The source for the tests along with build scripts
etc should probably be in a repo on their own.

What do you think?

Andreas

On 28/10/2016, 22:12, "gem5-dev on behalf of Alec Roelke"
 wrote:

>I think I get it.  Thanks for your help!
>
>It looks like I won't be able to make use of any tests other than
>00.hello,
>because either they're multithreaded or pieces of them are missing.  I'm
>going to try to put my instruction tests into 02.insttest.
>
>On Fri, Oct 28, 2016 at 11:01 AM, Jason Lowe-Power 
>wrote:
>
>> Hi Alec,
>>
>> Our regression testing infrastructure is confusing, to say the least.
>> Honestly, it would take me at least a few hours to figure out how to add
>> new regressions again. But here's a little information that hopefully
>>will
>> help you save some time.
>>
>> You need to put reference outputs in tests/quick/se/> NAME>/ref/riscv/linux/.
>>
>> The config files are in tests/configs. One of these files (without the
>> .py) is what goes in  above. The  can be
>> anything. Using some of the same tests that are there make sense, but
>>you
>> may want to add other RISC-V specific tests as well.
>>
>> You *may* be able to use the config files in tests/configs without
>> changes, but maybe not. I'm not sure. You'll probably be able to use
>>some
>> but not others.
>>
>> Getting scons to run your regressions is totally black magic to me.
>>Here's
>> a couple of things I've learned, though.
>> 1) You should delete the build directory (e.g., build/RISCV/tests/
>> debug/quick/se/00.hello/riscv/) every time you want to re-run the
>> regression.
>> 2) The regression won't even try to run if you don't have dummy files in
>> tests/quick/se//ref/riscv/linux/.
>>
>> For the missing stats/changed stats, since this is first time you're
>> running the RISC-V stats, I would totally ignore all of that
>>information. I
>> would just look at the output (simout/simerr) and make sure it seems
>> reasonable (e.g., no errors). If you have an idea of a stat that you
>>expect
>> to see (e.g., floating point ops > 0) then you could look at the stats
>>file
>> too to make sure it's what you expect.
>>
>> There's no specific configuration you should be using. For the most
>>part,
>> all of the tests are just functional tests. The configurations for the
>> system are specified in the tests/configs () Python
>>files.
>>
>> Hopefully this will help you get started. Let us know if you have more
>> questions. Maybe someone with more regression tester experience will be
>> able to help more.
>>
>> Jason
>>
>> On Thu, Oct 27, 2016 at 8:23 PM Alec Roelke  wrote:
>>
>>> I'll start with converting as many from "quick" as I can.  If/when I
>>>end
>>> up creating my own, is there a convention for what they should be
>>>named and
>>> where they should go?
>>>
>>> Also, I'm having a bit of a problem with just 00.hello.  I compiled the
>>> source in tests/test-progs/hello/src into
>>>tests/test-progs/hello/bin/riscv/linux
>>> (for some reason mine disappeared), and then created
>>> tests/quick/se/00.hello/ref/riscv/linux/simple-atomic.  Then I compiled
>>> and ran build/RISCV/gem5.fast, configuring for the atomic CPU and
>>> redirecting the output stats, configuration, stdout, and stderr into
>>>that
>>> directory.  But when I run build/RISCV/tests/debug/quick/
>>> se/00.hello/riscv/linux/simple-atomic, it claims that it couldn't find
>>> several stats and that it found several other unexpected ones in
>>>stats.txt
>>> (and completely ignored the other files).  Is there some other
>>> configuration I should be using to generate the output for the
>>>regression?
>>> Actually, better yet, is there a way for me to figure out what the
>>> configuration I should be using is, since I imagine I'll run into this
>>> problem for the other CPU models?
>>>
>>> On Thu, Oct 27, 2016 at 6:38 PM, Jason Lowe-Power 
>>> wrote:
>>>
>>> Hi Alec,
>>>
>>> Thanks for taking the time for writing tests. It's something that we
>>>as a
>>> community need to get better at.
>>>
>>> To respond to your questions:
>>> - It is completely acceptable for you to include RISC-V only tests. In
>>> fact, I think it's a necessity.
>>> - Focusing just on the "quick" regressions makes sense to me. If you
>>>have
>>> one or two longer benchmarks, that would be good, but not required. If
>>>you
>>> could stay away from SPEC it would be better. I 

Re: [gem5-dev] Status of RISC-V patches

[Alec Roelke]

I think I get it.  Thanks for your help!

It looks like I won't be able to make use of any tests other than 00.hello,
because either they're multithreaded or pieces of them are missing.  I'm
going to try to put my instruction tests into 02.insttest.

On Fri, Oct 28, 2016 at 11:01 AM, Jason Lowe-Power 
wrote:

> Hi Alec,
>
> Our regression testing infrastructure is confusing, to say the least.
> Honestly, it would take me at least a few hours to figure out how to add
> new regressions again. But here's a little information that hopefully will
> help you save some time.
>
> You need to put reference outputs in tests/quick/se/ NAME>/ref/riscv/linux/.
>
> The config files are in tests/configs. One of these files (without the
> .py) is what goes in  above. The  can be
> anything. Using some of the same tests that are there make sense, but you
> may want to add other RISC-V specific tests as well.
>
> You *may* be able to use the config files in tests/configs without
> changes, but maybe not. I'm not sure. You'll probably be able to use some
> but not others.
>
> Getting scons to run your regressions is totally black magic to me. Here's
> a couple of things I've learned, though.
> 1) You should delete the build directory (e.g., build/RISCV/tests/
> debug/quick/se/00.hello/riscv/) every time you want to re-run the
> regression.
> 2) The regression won't even try to run if you don't have dummy files in
> tests/quick/se//ref/riscv/linux/.
>
> For the missing stats/changed stats, since this is first time you're
> running the RISC-V stats, I would totally ignore all of that information. I
> would just look at the output (simout/simerr) and make sure it seems
> reasonable (e.g., no errors). If you have an idea of a stat that you expect
> to see (e.g., floating point ops > 0) then you could look at the stats file
> too to make sure it's what you expect.
>
> There's no specific configuration you should be using. For the most part,
> all of the tests are just functional tests. The configurations for the
> system are specified in the tests/configs () Python files.
>
> Hopefully this will help you get started. Let us know if you have more
> questions. Maybe someone with more regression tester experience will be
> able to help more.
>
> Jason
>
> On Thu, Oct 27, 2016 at 8:23 PM Alec Roelke  wrote:
>
>> I'll start with converting as many from "quick" as I can.  If/when I end
>> up creating my own, is there a convention for what they should be named and
>> where they should go?
>>
>> Also, I'm having a bit of a problem with just 00.hello.  I compiled the
>> source in tests/test-progs/hello/src into 
>> tests/test-progs/hello/bin/riscv/linux
>> (for some reason mine disappeared), and then created
>> tests/quick/se/00.hello/ref/riscv/linux/simple-atomic.  Then I compiled
>> and ran build/RISCV/gem5.fast, configuring for the atomic CPU and
>> redirecting the output stats, configuration, stdout, and stderr into that
>> directory.  But when I run build/RISCV/tests/debug/quick/
>> se/00.hello/riscv/linux/simple-atomic, it claims that it couldn't find
>> several stats and that it found several other unexpected ones in stats.txt
>> (and completely ignored the other files).  Is there some other
>> configuration I should be using to generate the output for the regression?
>> Actually, better yet, is there a way for me to figure out what the
>> configuration I should be using is, since I imagine I'll run into this
>> problem for the other CPU models?
>>
>> On Thu, Oct 27, 2016 at 6:38 PM, Jason Lowe-Power 
>> wrote:
>>
>> Hi Alec,
>>
>> Thanks for taking the time for writing tests. It's something that we as a
>> community need to get better at.
>>
>> To respond to your questions:
>> - It is completely acceptable for you to include RISC-V only tests. In
>> fact, I think it's a necessity.
>> - Focusing just on the "quick" regressions makes sense to me. If you have
>> one or two longer benchmarks, that would be good, but not required. If you
>> could stay away from SPEC it would be better. I believe that we can't
>> distribute the binaries, which makes it a pain for others to run the
>> regression test.
>> - Covering corner cases would be amazing, but again, not required. If you
>> look at gem5's current regression suite, you'll find that we currently
>> don't have anything like that. So, if it doesn't take you too much time,
>> this would be a good addition, but just getting coverage of all
>> instructions would be a step up from all the other ISAs.
>> - For m5threads, no need to implement it for RISC-V. Although, it looks
>> like you only need to implement 3 functions, so I don't think it would be
>> too hard. But that's a project for another day :).
>>
>> Again, I want to stress that we can't expect you to spend lots of time
>> writing great regressions. It would be very hypocritical :). Anything is
>> acceptable. Of course, better regressions mean that RISC-V will be more

Re: [gem5-dev] Status of RISC-V patches

[Jason Lowe-Power]

Hi Alec,

Our regression testing infrastructure is confusing, to say the least.
Honestly, it would take me at least a few hours to figure out how to add
new regressions again. But here's a little information that hopefully will
help you save some time.

You need to put reference outputs in tests/quick/se//ref/riscv/linux/.

The config files are in tests/configs. One of these files (without the .py)
is what goes in  above. The  can be anything.
Using some of the same tests that are there make sense, but you may want to
add other RISC-V specific tests as well.

You *may* be able to use the config files in tests/configs without changes,
but maybe not. I'm not sure. You'll probably be able to use some but not
others.

Getting scons to run your regressions is totally black magic to me. Here's
a couple of things I've learned, though.
1) You should delete the build directory
(e.g., build/RISCV/tests/debug/quick/se/00.hello/riscv/) every time you
want to re-run the regression.
2) The regression won't even try to run if you don't have dummy files in
tests/quick/se//ref/riscv/linux/.

For the missing stats/changed stats, since this is first time you're
running the RISC-V stats, I would totally ignore all of that information. I
would just look at the output (simout/simerr) and make sure it seems
reasonable (e.g., no errors). If you have an idea of a stat that you expect
to see (e.g., floating point ops > 0) then you could look at the stats file
too to make sure it's what you expect.

There's no specific configuration you should be using. For the most part,
all of the tests are just functional tests. The configurations for the
system are specified in the tests/configs () Python files.

Hopefully this will help you get started. Let us know if you have more
questions. Maybe someone with more regression tester experience will be
able to help more.

Jason

On Thu, Oct 27, 2016 at 8:23 PM Alec Roelke  wrote:

> I'll start with converting as many from "quick" as I can.  If/when I end
> up creating my own, is there a convention for what they should be named and
> where they should go?
>
> Also, I'm having a bit of a problem with just 00.hello.  I compiled the
> source in tests/test-progs/hello/src into
> tests/test-progs/hello/bin/riscv/linux (for some reason mine disappeared),
> and then created tests/quick/se/00.hello/ref/riscv/linux/simple-atomic.
> Then I compiled and ran build/RISCV/gem5.fast, configuring for the atomic
> CPU and redirecting the output stats, configuration, stdout, and stderr
> into that directory.  But when I run
> build/RISCV/tests/debug/quick/se/00.hello/riscv/linux/simple-atomic, it
> claims that it couldn't find several stats and that it found several other
> unexpected ones in stats.txt (and completely ignored the other files).  Is
> there some other configuration I should be using to generate the output for
> the regression?  Actually, better yet, is there a way for me to figure out
> what the configuration I should be using is, since I imagine I'll run into
> this problem for the other CPU models?
>
> On Thu, Oct 27, 2016 at 6:38 PM, Jason Lowe-Power 
> wrote:
>
> Hi Alec,
>
> Thanks for taking the time for writing tests. It's something that we as a
> community need to get better at.
>
> To respond to your questions:
> - It is completely acceptable for you to include RISC-V only tests. In
> fact, I think it's a necessity.
> - Focusing just on the "quick" regressions makes sense to me. If you have
> one or two longer benchmarks, that would be good, but not required. If you
> could stay away from SPEC it would be better. I believe that we can't
> distribute the binaries, which makes it a pain for others to run the
> regression test.
> - Covering corner cases would be amazing, but again, not required. If you
> look at gem5's current regression suite, you'll find that we currently
> don't have anything like that. So, if it doesn't take you too much time,
> this would be a good addition, but just getting coverage of all
> instructions would be a step up from all the other ISAs.
> - For m5threads, no need to implement it for RISC-V. Although, it looks
> like you only need to implement 3 functions, so I don't think it would be
> too hard. But that's a project for another day :).
>
> Again, I want to stress that we can't expect you to spend lots of time
> writing great regressions. It would be very hypocritical :). Anything is
> acceptable. Of course, better regressions mean that RISC-V will be more
> usable and more stable.
>
> Cheers,
> Jason
>
> On Thu, Oct 27, 2016 at 5:29 PM Alec Roelke  wrote:
>
> I'll certainly add regressions for "hello" for each of the four models,
> and I'll try to get other "quick" tests done the same way, too.  I won't be
> able to do all of them as m5threads hasn't been implemented for RISC-V, but
> I'll do what I can.  I can also do the "long" ones the same way, if time
> isn't a concern (I noticed some were 

Re: [gem5-dev] Status of RISC-V patches

[Alec Roelke]

I'll start with converting as many from "quick" as I can.  If/when I end up
creating my own, is there a convention for what they should be named and
where they should go?

Also, I'm having a bit of a problem with just 00.hello.  I compiled the
source in tests/test-progs/hello/src into
tests/test-progs/hello/bin/riscv/linux (for some reason mine disappeared),
and then created tests/quick/se/00.hello/ref/riscv/linux/simple-atomic.
Then I compiled and ran build/RISCV/gem5.fast, configuring for the atomic
CPU and redirecting the output stats, configuration, stdout, and stderr
into that directory.  But when I run
build/RISCV/tests/debug/quick/se/00.hello/riscv/linux/simple-atomic, it
claims that it couldn't find several stats and that it found several other
unexpected ones in stats.txt (and completely ignored the other files).  Is
there some other configuration I should be using to generate the output for
the regression?  Actually, better yet, is there a way for me to figure out
what the configuration I should be using is, since I imagine I'll run into
this problem for the other CPU models?

On Thu, Oct 27, 2016 at 6:38 PM, Jason Lowe-Power 
wrote:

> Hi Alec,
>
> Thanks for taking the time for writing tests. It's something that we as a
> community need to get better at.
>
> To respond to your questions:
> - It is completely acceptable for you to include RISC-V only tests. In
> fact, I think it's a necessity.
> - Focusing just on the "quick" regressions makes sense to me. If you have
> one or two longer benchmarks, that would be good, but not required. If you
> could stay away from SPEC it would be better. I believe that we can't
> distribute the binaries, which makes it a pain for others to run the
> regression test.
> - Covering corner cases would be amazing, but again, not required. If you
> look at gem5's current regression suite, you'll find that we currently
> don't have anything like that. So, if it doesn't take you too much time,
> this would be a good addition, but just getting coverage of all
> instructions would be a step up from all the other ISAs.
> - For m5threads, no need to implement it for RISC-V. Although, it looks
> like you only need to implement 3 functions, so I don't think it would be
> too hard. But that's a project for another day :).
>
> Again, I want to stress that we can't expect you to spend lots of time
> writing great regressions. It would be very hypocritical :). Anything is
> acceptable. Of course, better regressions mean that RISC-V will be more
> usable and more stable.
>
> Cheers,
> Jason
>
> On Thu, Oct 27, 2016 at 5:29 PM Alec Roelke  wrote:
>
>> I'll certainly add regressions for "hello" for each of the four models,
>> and I'll try to get other "quick" tests done the same way, too.  I won't be
>> able to do all of them as m5threads hasn't been implemented for RISC-V, but
>> I'll do what I can.  I can also do the "long" ones the same way, if time
>> isn't a concern (I noticed some were from SPEC, which could take a long
>> time to complete).
>>
>> Because m5threads hasn't been implemented for RISC-V, and my patches only
>> support SE mode, I can't actually test if the atomic instructions work
>> properly when used concurrently, but I can at least test that they perform
>> the read-modify-write operations properly.  Is it okay if I add a few
>> regressions that only work for RISC-V since they'd use assembly calls?  For
>> that matter, should I be making sure that the existing regressions cover
>> corner cases in instructions, or is it sufficient to see that each
>> instruction is represented at least once by them?  I could write some tests
>> that check corner cases, but at least some would use assembly calls and
>> thus be incompatible with anything other than RISC-V.
>>
>> On Thu, Oct 27, 2016 at 5:55 PM, Jason Lowe-Power 
>> wrote:
>>
>> Hi Alec,
>>
>> Thanks again for implementing RISC-V in gem5. It's an incredibly
>> important and timely addition!
>>
>> As far as I can tell, the patches look good. Hopefully some other will
>> review them soon as well.
>>
>> The only thing that's missing that I would really like to have before
>> pushing the patches is some regression tests for RISC-V. If you could look
>> at http://gem5.org/Regression_Tests and have a go at adding some
>> regressions, it would be helpful. It would be *great* if you could make
>> sure the regressions cover most of what you've implemented (e.g.,
>> multiply/atomic/etc. instructions, Linux syscalls, etc.). If that isn't
>> possible, at least having a "hello" regression for a couple of different
>> CPU models is needed.
>>
>> Thanks again for your contribution!
>>
>> Jason
>>
>>
>>
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] Status of RISC-V patches

[Alec Roelke]

I'll certainly add regressions for "hello" for each of the four models, and
I'll try to get other "quick" tests done the same way, too.  I won't be
able to do all of them as m5threads hasn't been implemented for RISC-V, but
I'll do what I can.  I can also do the "long" ones the same way, if time
isn't a concern (I noticed some were from SPEC, which could take a long
time to complete).

Because m5threads hasn't been implemented for RISC-V, and my patches only
support SE mode, I can't actually test if the atomic instructions work
properly when used concurrently, but I can at least test that they perform
the read-modify-write operations properly.  Is it okay if I add a few
regressions that only work for RISC-V since they'd use assembly calls?  For
that matter, should I be making sure that the existing regressions cover
corner cases in instructions, or is it sufficient to see that each
instruction is represented at least once by them?  I could write some tests
that check corner cases, but at least some would use assembly calls and
thus be incompatible with anything other than RISC-V.

On Thu, Oct 27, 2016 at 5:55 PM, Jason Lowe-Power 
wrote:

> Hi Alec,
>
> Thanks again for implementing RISC-V in gem5. It's an incredibly important
> and timely addition!
>
> As far as I can tell, the patches look good. Hopefully some other will
> review them soon as well.
>
> The only thing that's missing that I would really like to have before
> pushing the patches is some regression tests for RISC-V. If you could look
> at http://gem5.org/Regression_Tests and have a go at adding some
> regressions, it would be helpful. It would be *great* if you could make
> sure the regressions cover most of what you've implemented (e.g.,
> multiply/atomic/etc. instructions, Linux syscalls, etc.). If that isn't
> possible, at least having a "hello" regression for a couple of different
> CPU models is needed.
>
> Thanks again for your contribution!
>
> Jason
>
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

Re: [gem5-dev] Status of RISC-V patches

[Jason Lowe-Power]

Hi Alec,

Thanks for taking the time for writing tests. It's something that we as a
community need to get better at.

To respond to your questions:
- It is completely acceptable for you to include RISC-V only tests. In
fact, I think it's a necessity.
- Focusing just on the "quick" regressions makes sense to me. If you have
one or two longer benchmarks, that would be good, but not required. If you
could stay away from SPEC it would be better. I believe that we can't
distribute the binaries, which makes it a pain for others to run the
regression test.
- Covering corner cases would be amazing, but again, not required. If you
look at gem5's current regression suite, you'll find that we currently
don't have anything like that. So, if it doesn't take you too much time,
this would be a good addition, but just getting coverage of all
instructions would be a step up from all the other ISAs.
- For m5threads, no need to implement it for RISC-V. Although, it looks
like you only need to implement 3 functions, so I don't think it would be
too hard. But that's a project for another day :).

Again, I want to stress that we can't expect you to spend lots of time
writing great regressions. It would be very hypocritical :). Anything is
acceptable. Of course, better regressions mean that RISC-V will be more
usable and more stable.

Cheers,
Jason

On Thu, Oct 27, 2016 at 5:29 PM Alec Roelke  wrote:

> I'll certainly add regressions for "hello" for each of the four models,
> and I'll try to get other "quick" tests done the same way, too.  I won't be
> able to do all of them as m5threads hasn't been implemented for RISC-V, but
> I'll do what I can.  I can also do the "long" ones the same way, if time
> isn't a concern (I noticed some were from SPEC, which could take a long
> time to complete).
>
> Because m5threads hasn't been implemented for RISC-V, and my patches only
> support SE mode, I can't actually test if the atomic instructions work
> properly when used concurrently, but I can at least test that they perform
> the read-modify-write operations properly.  Is it okay if I add a few
> regressions that only work for RISC-V since they'd use assembly calls?  For
> that matter, should I be making sure that the existing regressions cover
> corner cases in instructions, or is it sufficient to see that each
> instruction is represented at least once by them?  I could write some tests
> that check corner cases, but at least some would use assembly calls and
> thus be incompatible with anything other than RISC-V.
>
> On Thu, Oct 27, 2016 at 5:55 PM, Jason Lowe-Power 
> wrote:
>
> Hi Alec,
>
> Thanks again for implementing RISC-V in gem5. It's an incredibly important
> and timely addition!
>
> As far as I can tell, the patches look good. Hopefully some other will
> review them soon as well.
>
> The only thing that's missing that I would really like to have before
> pushing the patches is some regression tests for RISC-V. If you could look
> at http://gem5.org/Regression_Tests and have a go at adding some
> regressions, it would be helpful. It would be *great* if you could make
> sure the regressions cover most of what you've implemented (e.g.,
> multiply/atomic/etc. instructions, Linux syscalls, etc.). If that isn't
> possible, at least having a "hello" regression for a couple of different
> CPU models is needed.
>
> Thanks again for your contribution!
>
> Jason
>
>
>
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] Status of RISC-V patches

[Jason Lowe-Power]

Hi Alec,

Thanks again for implementing RISC-V in gem5. It's an incredibly important
and timely addition!

As far as I can tell, the patches look good. Hopefully some other will
review them soon as well.

The only thing that's missing that I would really like to have before
pushing the patches is some regression tests for RISC-V. If you could look
at http://gem5.org/Regression_Tests and have a go at adding some
regressions, it would be helpful. It would be *great* if you could make
sure the regressions cover most of what you've implemented (e.g.,
multiply/atomic/etc. instructions, Linux syscalls, etc.). If that isn't
possible, at least having a "hello" regression for a couple of different
CPU models is needed.

Thanks again for your contribution!

Jason
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev