[gem5-dev] Cron /z/m5/regression/do-regression quick

2019-12-12 Thread Cron Daemon
* build/NULL/tests/opt/quick/se/51.memcheck/null/none/memcheck: FAILED!
* build/RISCV/tests/opt/quick/se/02.insttest/riscv/linux-rv64c/o3-timing: 
FAILED!
* 
build/RISCV/tests/opt/quick/se/02.insttest/riscv/linux-rv64c/simple-timing-ruby:
 FAILED!
* 
build/RISCV/tests/opt/quick/se/02.insttest/riscv/linux-rv64c/simple-atomic: 
FAILED!
* 
build/RISCV/tests/opt/quick/se/02.insttest/riscv/linux-rv64c/simple-timing: 
FAILED!
* 
build/RISCV/tests/opt/quick/se/02.insttest/riscv/linux-rv64c/minor-timing: 
FAILED!
* 
build/ALPHA/tests/opt/quick/se/03.learning-gem5/alpha/linux/learning-gem5-p1-two-level:
 CHANGED!
* build/ALPHA/tests/opt/quick/se/00.hello/alpha/linux/simple-timing: 
CHANGED!
* build/ALPHA/tests/opt/quick/se/00.hello/alpha/linux/simple-atomic: 
CHANGED!
* 
build/ALPHA/tests/opt/quick/se/03.learning-gem5/alpha/linux/learning-gem5-p1-simple:
 CHANGED!
* build/ALPHA/tests/opt/quick/se/00.hello/alpha/linux/simple-timing-ruby: 
CHANGED!
* build/ALPHA/tests/opt/quick/se/00.hello/alpha/linux/minor-timing: CHANGED!
* build/ALPHA/tests/opt/quick/se/01.hello-2T-smt/alpha/linux/o3-timing-mt: 
CHANGED!
* build/ALPHA/tests/opt/quick/se/00.hello/alpha/linux/o3-timing: CHANGED!
* 
build/ALPHA/tests/opt/quick/fs/10.linux-boot/alpha/linux/tsunami-simple-atomic: 
CHANGED!
* 
build/ALPHA/tests/opt/quick/fs/10.linux-boot/alpha/linux/tsunami-simple-atomic-dual:
 CHANGED!
* 
build/ALPHA/tests/opt/quick/fs/10.linux-boot/alpha/linux/tsunami-simple-timing: 
CHANGED!
* 
build/ALPHA/tests/opt/quick/fs/10.linux-boot/alpha/linux/tsunami-simple-timing-dual:
 CHANGED!
* 
build/ALPHA/tests/opt/quick/fs/80.netperf-stream/alpha/linux/twosys-tsunami-simple-atomic:
 CHANGED!
* 
build/MIPS/tests/opt/quick/se/03.learning-gem5/mips/linux/learning-gem5-p1-two-level:
 CHANGED!
* 
build/MIPS/tests/opt/quick/se/03.learning-gem5/mips/linux/learning-gem5-p1-simple:
 CHANGED!
* build/MIPS/tests/opt/quick/se/00.hello/mips/linux/simple-timing-ruby: 
CHANGED!
* build/MIPS/tests/opt/quick/se/00.hello/mips/linux/simple-timing: CHANGED!
* build/MIPS/tests/opt/quick/se/00.hello/mips/linux/simple-atomic: CHANGED!
* build/MIPS/tests/opt/quick/se/00.hello/mips/linux/o3-timing: CHANGED!
* build/NULL/tests/opt/quick/se/80.dram-openpage/null/none/dram-lowp: 
CHANGED!
* build/NULL/tests/opt/quick/se/80.dram-closepage/null/none/dram-lowp: 
CHANGED!
* build/NULL/tests/opt/quick/se/70.tgen/null/none/tgen-simple-mem: CHANGED!
* build/NULL/tests/opt/quick/se/60.rubytest/null/none/rubytest-ruby: 
CHANGED!
* build/NULL/tests/opt/quick/se/70.tgen/null/none/tgen-dram-ctrl: CHANGED!
* 
build/NULL_MOESI_hammer/tests/opt/quick/se/60.rubytest/null/none/rubytest-ruby-MOESI_hammer:
 CHANGED!
* 
build/NULL_MESI_Two_Level/tests/opt/quick/se/60.rubytest/null/none/rubytest-ruby-MESI_Two_Level:
 CHANGED!
* 
build/NULL_MOESI_CMP_directory/tests/opt/quick/se/60.rubytest/null/none/rubytest-ruby-MOESI_CMP_directory:
 CHANGED!
* 
build/NULL_MOESI_CMP_token/tests/opt/quick/se/60.rubytest/null/none/rubytest-ruby-MOESI_CMP_token:
 CHANGED!
* build/POWER/tests/opt/quick/se/00.hello/power/linux/simple-atomic: 
CHANGED!
* build/POWER/tests/opt/quick/se/00.hello/power/linux/o3-timing: CHANGED!
* build/SPARC/tests/opt/quick/se/00.hello/sparc/linux/simple-atomic: 
CHANGED!
* 
build/SPARC/tests/opt/quick/se/03.learning-gem5/sparc/linux/learning-gem5-p1-simple:
 CHANGED!
* build/SPARC/tests/opt/quick/se/00.hello/sparc/linux/simple-timing: 
CHANGED!
* 
build/SPARC/tests/opt/quick/se/40.m5threads-test-atomic/sparc/linux/simple-atomic-mp:
 CHANGED!
* build/SPARC/tests/opt/quick/se/02.insttest/sparc/linux/simple-timing: 
CHANGED!
* 
build/SPARC/tests/opt/quick/se/40.m5threads-test-atomic/sparc/linux/simple-timing-mp:
 CHANGED!
* build/SPARC/tests/opt/quick/se/00.hello/sparc/linux/simple-timing-ruby: 
CHANGED!
* build/SPARC/tests/opt/quick/se/02.insttest/sparc/linux/simple-atomic: 
CHANGED!
* build/SPARC/tests/opt/quick/se/02.insttest/sparc/linux/o3-timing: CHANGED!
* 
build/SPARC/tests/opt/quick/se/03.learning-gem5/sparc/linux/learning-gem5-p1-two-level:
 CHANGED!
* 
build/SPARC/tests/opt/quick/se/40.m5threads-test-atomic/sparc/linux/o3-timing-mp:
 CHANGED!
* build/SPARC/tests/opt/quick/se/50.vortex/sparc/linux/simple-atomic: 
CHANGED!
* build/SPARC/tests/opt/quick/se/70.twolf/sparc/linux/simple-atomic: 
CHANGED!
* build/SPARC/tests/opt/quick/se/10.mcf/sparc/linux/simple-atomic: CHANGED!
* build/SPARC/tests/opt/quick/se/50.vortex/sparc/linux/simple-timing: 
CHANGED!
* build/SPARC/tests/opt/quick/se/70.twolf/sparc/linux/simple-timing: 
CHANGED!
* build/X86/tests/opt/quick/se/00.hello/x86/linux/simple-timing: CHANGED!
* 
build/X86/tests/opt/quick/se/03.learning-gem5/x86/linux/learning-gem5-p1-two-level:
 CHANGED!
* build/X86/tests/opt/quick/se/00.hello/x86/linux/simple-atomic: 

Re: [gem5-dev] gem5 19.0.0 : New git branching proposal

2019-12-12 Thread Gabe Black
I didn't see anything in the email above about minor vs. major releases.
Was that described somewhere else?

As far as publicizing API deprecation, that could be something we patch
into the preceding release some window before the new release. That could
actually be a good point to grab a snapshot of the development branch so it
can be tested and stabilized (ie fixed) before being released, since at
that point we would know what APIs were changing.

It would probably also be a good idea to send a heads up email to the user
mailing list at that time since people probably won't pull down new
versions of the releases very often, particularly if they don't change much
(which is they point after all).

Gabe

On Thu, Dec 12, 2019 at 5:16 PM Jason Lowe-Power 
wrote:

> Ok... so are you suggesting there be no difference between major and minor
> releases?
>
> How do you suggest communicating that API changes are coming? The most
> common way to do this is by marking functions as deprecated for some period
> of time.
>
> Thanks,
> Jason
>
> On Thu, Dec 12, 2019 at 4:26 PM Gabe Black  wrote:
>
> > I think that's what the release vs. development branch split already
> > provides. If you want stable interfaces, then the release branch will
> stay
> > as it is for a year, or forever if you don't move to new releases.
> >
> > Gabe
> >
> > On Thu, Dec 12, 2019 at 4:18 PM Jason Lowe-Power 
> > wrote:
> >
> > > Thanks Gabe!
> > >
> > > I think we're mostly in agreement. The key disagreement is the
> following:
> > >
> > > Note that I'm not advocating for a wild west free for all with no
> > > > accountability, but only being able to make changes to "interfaces"
> > > (which
> > > > pretty much everything could be considered) once a year is much too
> far
> > > in
> > > > the other direction, especially on a so called development branch.
> > >
> > >
> > > I believe, and I've heard from many gem5 users, that we need to have
> more
> > > stable interfaces. I think that once a year is a good compromise, but
> I'm
> > > open to a faster cadence. Either way, we must give a heads up of at
> least
> > > months to change an interface.
> > >
> > > Can you propose a different idea that will satisfy our need to both
> > > provided stability and allow developers to update code?
> > >
> > > Alternatively, maybe we need to come up with a better transition than
> > just
> > > "tomorrow you can't modify interfaces." Do you have any ideas on a more
> > > smooth transition to this development model of providing stability to
> our
> > > users?
> > >
> > > Thanks,
> > > Jason
> > >
> > > On Thu, Dec 12, 2019 at 4:03 PM Gabe Black 
> wrote:
> > >
> > > > I'll point out that nobody had to rebase O3 when, for example, the
> > > > ExecContext changed, because it was already in the tree and was
> changed
> > > > along with everything else.
> > > >
> > > > I think you have an important contradiction in your reasoning above,
> > > namely
> > > > that gem5 is simultaneously too unstable and too stable. You're
> saying
> > > it's
> > > > too unstable since the interfaces change too often, but then also say
> > > that
> > > > there have been very few new features in gem5 in the last few years.
> > > >
> > > > Fundamentally, new features require changing things. If we clamp down
> > on
> > > > changing what exists or add a bunch of onerous requirements, then
> there
> > > > will either be no change, or it will be side stepped by laying down a
> > > layer
> > > > of concrete and adding something new over the top, ie ifdefs, config
> > > > options, "real" versions of APIs which live alongside the old ones.
> > These
> > > > are all things which have already happened in gem5 and which I've
> spent
> > > > some time partially cleaning up. In the real world, see the x86 ISA
> as
> > an
> > > > example. Do I really need to be able to boot my huge workstation into
> > > > little bitty 16 bit real mode? Probably not, but I still could, and
> the
> > > ISA
> > > > is a bit of a mess because of it.
> > > >
> > > > Frankly, there is also a lot of technical debt in gem5. A lot of
> things
> > > > were accreted over time, often by well meaning but not necessary
> expert
> > > > programmers, including earlier (and even current) versions of me, and
> > > those
> > > > things need to be cleaned up. Fixing those things requires changing
> > > > bad/obsolete/etc interfaces, restructuring code, and generally moving
> > > > things around which have ripple effects throughout the simulator. You
> > > can't
> > > > make lemonade without cracking a few lemons.
> > > >
> > > > I think a few things are key:
> > > > Tests make changes less dangerous. We need more of these.
> > > > Warn people about changes so they have an opportunity to update and
> > don't
> > > > get blindsided.
> > > > Change interfaces carefully, with consideration for the future so
> they
> > > > don't have to be changed excessively.
> > > >
> > > > Note that I'm not advocating for a wild west free for all 

Re: [gem5-dev] gem5 19.0.0 : New git branching proposal

2019-12-12 Thread Jason Lowe-Power
Ok... so are you suggesting there be no difference between major and minor
releases?

How do you suggest communicating that API changes are coming? The most
common way to do this is by marking functions as deprecated for some period
of time.

Thanks,
Jason

On Thu, Dec 12, 2019 at 4:26 PM Gabe Black  wrote:

> I think that's what the release vs. development branch split already
> provides. If you want stable interfaces, then the release branch will stay
> as it is for a year, or forever if you don't move to new releases.
>
> Gabe
>
> On Thu, Dec 12, 2019 at 4:18 PM Jason Lowe-Power 
> wrote:
>
> > Thanks Gabe!
> >
> > I think we're mostly in agreement. The key disagreement is the following:
> >
> > Note that I'm not advocating for a wild west free for all with no
> > > accountability, but only being able to make changes to "interfaces"
> > (which
> > > pretty much everything could be considered) once a year is much too far
> > in
> > > the other direction, especially on a so called development branch.
> >
> >
> > I believe, and I've heard from many gem5 users, that we need to have more
> > stable interfaces. I think that once a year is a good compromise, but I'm
> > open to a faster cadence. Either way, we must give a heads up of at least
> > months to change an interface.
> >
> > Can you propose a different idea that will satisfy our need to both
> > provided stability and allow developers to update code?
> >
> > Alternatively, maybe we need to come up with a better transition than
> just
> > "tomorrow you can't modify interfaces." Do you have any ideas on a more
> > smooth transition to this development model of providing stability to our
> > users?
> >
> > Thanks,
> > Jason
> >
> > On Thu, Dec 12, 2019 at 4:03 PM Gabe Black  wrote:
> >
> > > I'll point out that nobody had to rebase O3 when, for example, the
> > > ExecContext changed, because it was already in the tree and was changed
> > > along with everything else.
> > >
> > > I think you have an important contradiction in your reasoning above,
> > namely
> > > that gem5 is simultaneously too unstable and too stable. You're saying
> > it's
> > > too unstable since the interfaces change too often, but then also say
> > that
> > > there have been very few new features in gem5 in the last few years.
> > >
> > > Fundamentally, new features require changing things. If we clamp down
> on
> > > changing what exists or add a bunch of onerous requirements, then there
> > > will either be no change, or it will be side stepped by laying down a
> > layer
> > > of concrete and adding something new over the top, ie ifdefs, config
> > > options, "real" versions of APIs which live alongside the old ones.
> These
> > > are all things which have already happened in gem5 and which I've spent
> > > some time partially cleaning up. In the real world, see the x86 ISA as
> an
> > > example. Do I really need to be able to boot my huge workstation into
> > > little bitty 16 bit real mode? Probably not, but I still could, and the
> > ISA
> > > is a bit of a mess because of it.
> > >
> > > Frankly, there is also a lot of technical debt in gem5. A lot of things
> > > were accreted over time, often by well meaning but not necessary expert
> > > programmers, including earlier (and even current) versions of me, and
> > those
> > > things need to be cleaned up. Fixing those things requires changing
> > > bad/obsolete/etc interfaces, restructuring code, and generally moving
> > > things around which have ripple effects throughout the simulator. You
> > can't
> > > make lemonade without cracking a few lemons.
> > >
> > > I think a few things are key:
> > > Tests make changes less dangerous. We need more of these.
> > > Warn people about changes so they have an opportunity to update and
> don't
> > > get blindsided.
> > > Change interfaces carefully, with consideration for the future so they
> > > don't have to be changed excessively.
> > >
> > > Note that I'm not advocating for a wild west free for all with no
> > > accountability, but only being able to make changes to "interfaces"
> > (which
> > > pretty much everything could be considered) once a year is much too far
> > in
> > > the other direction, especially on a so called development branch.
> > >
> > > Also, I think everyone here is well intentioned and wants to make gem5
> > > available to as many people as possible and as useful to them as
> > possible,
> > > and we're just trying to figure out how to do that. That's why we're
> > > hashing this out on a public mailing list and not in a private thread,
> > > secret meetings, etc.
> > >
> > > Gabe
> > >
> > > On Thu, Dec 12, 2019 at 10:08 AM Jason Lowe-Power  >
> > > wrote:
> > >
> > > > Hey Abhishek,
> > > >
> > > > Yes! Not only will we be releasing disk images (that we can depending
> > on
> > > > license for the benchmarks) and kernel images, we will also be
> > releasing
> > > > all of the documentation and the scripts describing how these images
> > were
> > > > 

Re: [gem5-dev] gem5 19.0.0 : New git branching proposal

2019-12-12 Thread Gabe Black
I think that's what the release vs. development branch split already
provides. If you want stable interfaces, then the release branch will stay
as it is for a year, or forever if you don't move to new releases.

Gabe

On Thu, Dec 12, 2019 at 4:18 PM Jason Lowe-Power 
wrote:

> Thanks Gabe!
>
> I think we're mostly in agreement. The key disagreement is the following:
>
> Note that I'm not advocating for a wild west free for all with no
> > accountability, but only being able to make changes to "interfaces"
> (which
> > pretty much everything could be considered) once a year is much too far
> in
> > the other direction, especially on a so called development branch.
>
>
> I believe, and I've heard from many gem5 users, that we need to have more
> stable interfaces. I think that once a year is a good compromise, but I'm
> open to a faster cadence. Either way, we must give a heads up of at least
> months to change an interface.
>
> Can you propose a different idea that will satisfy our need to both
> provided stability and allow developers to update code?
>
> Alternatively, maybe we need to come up with a better transition than just
> "tomorrow you can't modify interfaces." Do you have any ideas on a more
> smooth transition to this development model of providing stability to our
> users?
>
> Thanks,
> Jason
>
> On Thu, Dec 12, 2019 at 4:03 PM Gabe Black  wrote:
>
> > I'll point out that nobody had to rebase O3 when, for example, the
> > ExecContext changed, because it was already in the tree and was changed
> > along with everything else.
> >
> > I think you have an important contradiction in your reasoning above,
> namely
> > that gem5 is simultaneously too unstable and too stable. You're saying
> it's
> > too unstable since the interfaces change too often, but then also say
> that
> > there have been very few new features in gem5 in the last few years.
> >
> > Fundamentally, new features require changing things. If we clamp down on
> > changing what exists or add a bunch of onerous requirements, then there
> > will either be no change, or it will be side stepped by laying down a
> layer
> > of concrete and adding something new over the top, ie ifdefs, config
> > options, "real" versions of APIs which live alongside the old ones. These
> > are all things which have already happened in gem5 and which I've spent
> > some time partially cleaning up. In the real world, see the x86 ISA as an
> > example. Do I really need to be able to boot my huge workstation into
> > little bitty 16 bit real mode? Probably not, but I still could, and the
> ISA
> > is a bit of a mess because of it.
> >
> > Frankly, there is also a lot of technical debt in gem5. A lot of things
> > were accreted over time, often by well meaning but not necessary expert
> > programmers, including earlier (and even current) versions of me, and
> those
> > things need to be cleaned up. Fixing those things requires changing
> > bad/obsolete/etc interfaces, restructuring code, and generally moving
> > things around which have ripple effects throughout the simulator. You
> can't
> > make lemonade without cracking a few lemons.
> >
> > I think a few things are key:
> > Tests make changes less dangerous. We need more of these.
> > Warn people about changes so they have an opportunity to update and don't
> > get blindsided.
> > Change interfaces carefully, with consideration for the future so they
> > don't have to be changed excessively.
> >
> > Note that I'm not advocating for a wild west free for all with no
> > accountability, but only being able to make changes to "interfaces"
> (which
> > pretty much everything could be considered) once a year is much too far
> in
> > the other direction, especially on a so called development branch.
> >
> > Also, I think everyone here is well intentioned and wants to make gem5
> > available to as many people as possible and as useful to them as
> possible,
> > and we're just trying to figure out how to do that. That's why we're
> > hashing this out on a public mailing list and not in a private thread,
> > secret meetings, etc.
> >
> > Gabe
> >
> > On Thu, Dec 12, 2019 at 10:08 AM Jason Lowe-Power 
> > wrote:
> >
> > > Hey Abhishek,
> > >
> > > Yes! Not only will we be releasing disk images (that we can depending
> on
> > > license for the benchmarks) and kernel images, we will also be
> releasing
> > > all of the documentation and the scripts describing how these images
> were
> > > created. These will also be updated at every release as appropriate.
> > >
> > > Cheers,
> > > Jason
> > >
> > >
> > >
> > > On Thu, Dec 12, 2019 at 10:04 AM Abhishek Singh <
> > > abhishek.singh199...@gmail.com> wrote:
> > >
> > > > Hello Jason,
> > > >
> > > > This is perfect !
> > > >
> > > > I had one more question, for full system simulations will there be
> > > release
> > > > of images and kernel files for every architectures (arm , x86, etc)?
> > > >
> > > > On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power <
> 

Re: [gem5-dev] gem5 19.0.0 : New git branching proposal

2019-12-12 Thread Jason Lowe-Power
Thanks Gabe!

I think we're mostly in agreement. The key disagreement is the following:

Note that I'm not advocating for a wild west free for all with no
> accountability, but only being able to make changes to "interfaces" (which
> pretty much everything could be considered) once a year is much too far in
> the other direction, especially on a so called development branch.


I believe, and I've heard from many gem5 users, that we need to have more
stable interfaces. I think that once a year is a good compromise, but I'm
open to a faster cadence. Either way, we must give a heads up of at least
months to change an interface.

Can you propose a different idea that will satisfy our need to both
provided stability and allow developers to update code?

Alternatively, maybe we need to come up with a better transition than just
"tomorrow you can't modify interfaces." Do you have any ideas on a more
smooth transition to this development model of providing stability to our
users?

Thanks,
Jason

On Thu, Dec 12, 2019 at 4:03 PM Gabe Black  wrote:

> I'll point out that nobody had to rebase O3 when, for example, the
> ExecContext changed, because it was already in the tree and was changed
> along with everything else.
>
> I think you have an important contradiction in your reasoning above, namely
> that gem5 is simultaneously too unstable and too stable. You're saying it's
> too unstable since the interfaces change too often, but then also say that
> there have been very few new features in gem5 in the last few years.
>
> Fundamentally, new features require changing things. If we clamp down on
> changing what exists or add a bunch of onerous requirements, then there
> will either be no change, or it will be side stepped by laying down a layer
> of concrete and adding something new over the top, ie ifdefs, config
> options, "real" versions of APIs which live alongside the old ones. These
> are all things which have already happened in gem5 and which I've spent
> some time partially cleaning up. In the real world, see the x86 ISA as an
> example. Do I really need to be able to boot my huge workstation into
> little bitty 16 bit real mode? Probably not, but I still could, and the ISA
> is a bit of a mess because of it.
>
> Frankly, there is also a lot of technical debt in gem5. A lot of things
> were accreted over time, often by well meaning but not necessary expert
> programmers, including earlier (and even current) versions of me, and those
> things need to be cleaned up. Fixing those things requires changing
> bad/obsolete/etc interfaces, restructuring code, and generally moving
> things around which have ripple effects throughout the simulator. You can't
> make lemonade without cracking a few lemons.
>
> I think a few things are key:
> Tests make changes less dangerous. We need more of these.
> Warn people about changes so they have an opportunity to update and don't
> get blindsided.
> Change interfaces carefully, with consideration for the future so they
> don't have to be changed excessively.
>
> Note that I'm not advocating for a wild west free for all with no
> accountability, but only being able to make changes to "interfaces" (which
> pretty much everything could be considered) once a year is much too far in
> the other direction, especially on a so called development branch.
>
> Also, I think everyone here is well intentioned and wants to make gem5
> available to as many people as possible and as useful to them as possible,
> and we're just trying to figure out how to do that. That's why we're
> hashing this out on a public mailing list and not in a private thread,
> secret meetings, etc.
>
> Gabe
>
> On Thu, Dec 12, 2019 at 10:08 AM Jason Lowe-Power 
> wrote:
>
> > Hey Abhishek,
> >
> > Yes! Not only will we be releasing disk images (that we can depending on
> > license for the benchmarks) and kernel images, we will also be releasing
> > all of the documentation and the scripts describing how these images were
> > created. These will also be updated at every release as appropriate.
> >
> > Cheers,
> > Jason
> >
> >
> >
> > On Thu, Dec 12, 2019 at 10:04 AM Abhishek Singh <
> > abhishek.singh199...@gmail.com> wrote:
> >
> > > Hello Jason,
> > >
> > > This is perfect !
> > >
> > > I had one more question, for full system simulations will there be
> > release
> > > of images and kernel files for every architectures (arm , x86, etc)?
> > >
> > > On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power  >
> > > wrote:
> > >
> > > > Hey Abhishek,
> > > >
> > > > Emails will continue to gem5-dev on every changeset pushed to the
> > develop
> > > > branch as they do now to master :). We can discuss if we want the
> same
> > > for
> > > > feature branches (if we end up using them).
> > > >
> > > > Your first interpretation on gem5 stable is correct (sorry if this
> > wasn't
> > > > clear). It will be much more heavily tested than the minute-by-minute
> > > > releases from the develop branch. With this testing we will be
> 

Re: [gem5-dev] gem5 19.0.0 : New git branching proposal

2019-12-12 Thread Gabe Black
I'll point out that nobody had to rebase O3 when, for example, the
ExecContext changed, because it was already in the tree and was changed
along with everything else.

I think you have an important contradiction in your reasoning above, namely
that gem5 is simultaneously too unstable and too stable. You're saying it's
too unstable since the interfaces change too often, but then also say that
there have been very few new features in gem5 in the last few years.

Fundamentally, new features require changing things. If we clamp down on
changing what exists or add a bunch of onerous requirements, then there
will either be no change, or it will be side stepped by laying down a layer
of concrete and adding something new over the top, ie ifdefs, config
options, "real" versions of APIs which live alongside the old ones. These
are all things which have already happened in gem5 and which I've spent
some time partially cleaning up. In the real world, see the x86 ISA as an
example. Do I really need to be able to boot my huge workstation into
little bitty 16 bit real mode? Probably not, but I still could, and the ISA
is a bit of a mess because of it.

Frankly, there is also a lot of technical debt in gem5. A lot of things
were accreted over time, often by well meaning but not necessary expert
programmers, including earlier (and even current) versions of me, and those
things need to be cleaned up. Fixing those things requires changing
bad/obsolete/etc interfaces, restructuring code, and generally moving
things around which have ripple effects throughout the simulator. You can't
make lemonade without cracking a few lemons.

I think a few things are key:
Tests make changes less dangerous. We need more of these.
Warn people about changes so they have an opportunity to update and don't
get blindsided.
Change interfaces carefully, with consideration for the future so they
don't have to be changed excessively.

Note that I'm not advocating for a wild west free for all with no
accountability, but only being able to make changes to "interfaces" (which
pretty much everything could be considered) once a year is much too far in
the other direction, especially on a so called development branch.

Also, I think everyone here is well intentioned and wants to make gem5
available to as many people as possible and as useful to them as possible,
and we're just trying to figure out how to do that. That's why we're
hashing this out on a public mailing list and not in a private thread,
secret meetings, etc.

Gabe

On Thu, Dec 12, 2019 at 10:08 AM Jason Lowe-Power 
wrote:

> Hey Abhishek,
>
> Yes! Not only will we be releasing disk images (that we can depending on
> license for the benchmarks) and kernel images, we will also be releasing
> all of the documentation and the scripts describing how these images were
> created. These will also be updated at every release as appropriate.
>
> Cheers,
> Jason
>
>
>
> On Thu, Dec 12, 2019 at 10:04 AM Abhishek Singh <
> abhishek.singh199...@gmail.com> wrote:
>
> > Hello Jason,
> >
> > This is perfect !
> >
> > I had one more question, for full system simulations will there be
> release
> > of images and kernel files for every architectures (arm , x86, etc)?
> >
> > On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power 
> > wrote:
> >
> > > Hey Abhishek,
> > >
> > > Emails will continue to gem5-dev on every changeset pushed to the
> develop
> > > branch as they do now to master :). We can discuss if we want the same
> > for
> > > feature branches (if we end up using them).
> > >
> > > Your first interpretation on gem5 stable is correct (sorry if this
> wasn't
> > > clear). It will be much more heavily tested than the minute-by-minute
> > > releases from the develop branch. With this testing we will be
> publishing
> > > the following:
> > > - What (common) workloads are supported (e.g., SPEC, parsec, etc.).
> Which
> > > workloads we use here will be discussed in the future, stay tuned.
> > > - For all of the workloads, we will publish common statistics for a few
> > > different systems (e.g., time to simulate, IPC, cache miss rates,
> memory
> > > bandwidth, etc). The systems used and the stats will be discussed in
> the
> > > future.
> > >
> > > The stable release will *not* be a continuous process. The only time
> the
> > > stable branch will be updated is 1) At releases or 2) if a "major" bug
> is
> > > encountered. For non-release updates (e.g., for bugs), we'll be very
> > > careful to either re-validate our results or somehow know the results
> > won't
> > > change.
> > >
> > > Cheers,
> > > Jason
> > >
> > >
> > >
> > > On Thu, Dec 12, 2019 at 9:38 AM Abhishek Singh <
> > > abhishek.singh199...@gmail.com> wrote:
> > >
> > > > Hi Jason,
> > > >
> > > > Thanks for your email. It cleared a lot of misunderstanding which I
> > had.
> > > >
> > > > Is it possible to have an email sent on every commit we make to
> atleast
> > > > gem5-dev?
> > > > The email list could be different and can be sent to people 

Re: [gem5-dev] gem5 19.0.0 : New git branching proposal

2019-12-12 Thread Jason Lowe-Power
Hey Abhishek,

Yes! Not only will we be releasing disk images (that we can depending on
license for the benchmarks) and kernel images, we will also be releasing
all of the documentation and the scripts describing how these images were
created. These will also be updated at every release as appropriate.

Cheers,
Jason



On Thu, Dec 12, 2019 at 10:04 AM Abhishek Singh <
abhishek.singh199...@gmail.com> wrote:

> Hello Jason,
>
> This is perfect !
>
> I had one more question, for full system simulations will there be release
> of images and kernel files for every architectures (arm , x86, etc)?
>
> On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power 
> wrote:
>
> > Hey Abhishek,
> >
> > Emails will continue to gem5-dev on every changeset pushed to the develop
> > branch as they do now to master :). We can discuss if we want the same
> for
> > feature branches (if we end up using them).
> >
> > Your first interpretation on gem5 stable is correct (sorry if this wasn't
> > clear). It will be much more heavily tested than the minute-by-minute
> > releases from the develop branch. With this testing we will be publishing
> > the following:
> > - What (common) workloads are supported (e.g., SPEC, parsec, etc.). Which
> > workloads we use here will be discussed in the future, stay tuned.
> > - For all of the workloads, we will publish common statistics for a few
> > different systems (e.g., time to simulate, IPC, cache miss rates, memory
> > bandwidth, etc). The systems used and the stats will be discussed in the
> > future.
> >
> > The stable release will *not* be a continuous process. The only time the
> > stable branch will be updated is 1) At releases or 2) if a "major" bug is
> > encountered. For non-release updates (e.g., for bugs), we'll be very
> > careful to either re-validate our results or somehow know the results
> won't
> > change.
> >
> > Cheers,
> > Jason
> >
> >
> >
> > On Thu, Dec 12, 2019 at 9:38 AM Abhishek Singh <
> > abhishek.singh199...@gmail.com> wrote:
> >
> > > Hi Jason,
> > >
> > > Thanks for your email. It cleared a lot of misunderstanding which I
> had.
> > >
> > > Is it possible to have an email sent on every commit we make to atleast
> > > gem5-dev?
> > > The email list could be different and can be sent to people who are
> > > interested in this so that it does not spam to gem5-Dev list.
> > >
> > > I am talking about gem5 developer branch and not stable.
> > >
> > > This is because, it will keep all the interested community members well
> > > informed about new features that are added, who like to keep note of
> > latest
> > > changes and merge in their projects as required.
> > >
> > > And the way, I understand that a stable gem5 when a user desires is:
> > > It wants the conference accepted applications (standard workload for
> > > example spec, parsec, etc to run without any errors on both SE and FS
> > mode
> > > for every architecture and cpu models.
> > >
> > > If that’s what the stable releases are going to be testing before
> > releasing
> > > it, then a stable release is much more helpful and will have the wide
> > > reach. But when I read proposal this seems to be step by step process
> > > reaching it as the final goal. Please correct me on this if I
> understood
> > it
> > > wrong.
> > >
> > >
> > > The aim of stable releases is a continuous process and will always be
> > > doubtful unless tested with all the major conference accepted
> application
> > > on every stable releases.
> > >
> > > On Thu, Dec 12, 2019 at 12:12 PM Jason Lowe-Power  >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > First of all, let me say that I hope it's clear that gem5 is not
> > > > "controlled" in any way by me! As laid out in our governance
> document (
> > > > http://gem5.org/Governance#Overview), "gem5 is a meritocratic,
> > > > consensus-based community project". Through these emails, and as the
> > > chair
> > > > of the project management committee, I'm trying to *build* consensus.
> > > >
> > > > Importantly, there's a reason we're trying to make this push for
> stable
> > > > APIs. I've heard from dozens of current and potential gem5 users that
> > > they
> > > > want stable gem5 releases. By providing stable releases, we will be
> > > > expanding the users of gem5 (and, IMO, making the research and papers
> > > that
> > > > use gem5 better).
> > > >
> > > > Could you please clarify the policy on breaking APIs? It makes sense
> > for
> > > > > releases to maintain stable APIs, but how does that apply to the
> > > > > development branch? I'm worried that it will be very hard to make
> > > changes
> > > > > that don't change any interfaces, and we definitely don't want to
> > > > encourage
> > > > > a style of development where we just add and add and add without
> ever
> > > > > refactoring or consolidating things. If APIs can continue to change
> > as
> > > > > needed in the development branch and we just need to warn people
> > before
> > > > > they're released, that seems 

Re: [gem5-dev] gem5 19.0.0 : New git branching proposal

2019-12-12 Thread Abhishek Singh
Hello Jason,

This is perfect !

I had one more question, for full system simulations will there be release
of images and kernel files for every architectures (arm , x86, etc)?

On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power 
wrote:

> Hey Abhishek,
>
> Emails will continue to gem5-dev on every changeset pushed to the develop
> branch as they do now to master :). We can discuss if we want the same for
> feature branches (if we end up using them).
>
> Your first interpretation on gem5 stable is correct (sorry if this wasn't
> clear). It will be much more heavily tested than the minute-by-minute
> releases from the develop branch. With this testing we will be publishing
> the following:
> - What (common) workloads are supported (e.g., SPEC, parsec, etc.). Which
> workloads we use here will be discussed in the future, stay tuned.
> - For all of the workloads, we will publish common statistics for a few
> different systems (e.g., time to simulate, IPC, cache miss rates, memory
> bandwidth, etc). The systems used and the stats will be discussed in the
> future.
>
> The stable release will *not* be a continuous process. The only time the
> stable branch will be updated is 1) At releases or 2) if a "major" bug is
> encountered. For non-release updates (e.g., for bugs), we'll be very
> careful to either re-validate our results or somehow know the results won't
> change.
>
> Cheers,
> Jason
>
>
>
> On Thu, Dec 12, 2019 at 9:38 AM Abhishek Singh <
> abhishek.singh199...@gmail.com> wrote:
>
> > Hi Jason,
> >
> > Thanks for your email. It cleared a lot of misunderstanding which I had.
> >
> > Is it possible to have an email sent on every commit we make to atleast
> > gem5-dev?
> > The email list could be different and can be sent to people who are
> > interested in this so that it does not spam to gem5-Dev list.
> >
> > I am talking about gem5 developer branch and not stable.
> >
> > This is because, it will keep all the interested community members well
> > informed about new features that are added, who like to keep note of
> latest
> > changes and merge in their projects as required.
> >
> > And the way, I understand that a stable gem5 when a user desires is:
> > It wants the conference accepted applications (standard workload for
> > example spec, parsec, etc to run without any errors on both SE and FS
> mode
> > for every architecture and cpu models.
> >
> > If that’s what the stable releases are going to be testing before
> releasing
> > it, then a stable release is much more helpful and will have the wide
> > reach. But when I read proposal this seems to be step by step process
> > reaching it as the final goal. Please correct me on this if I understood
> it
> > wrong.
> >
> >
> > The aim of stable releases is a continuous process and will always be
> > doubtful unless tested with all the major conference accepted application
> > on every stable releases.
> >
> > On Thu, Dec 12, 2019 at 12:12 PM Jason Lowe-Power 
> > wrote:
> >
> > > Hi all,
> > >
> > > First of all, let me say that I hope it's clear that gem5 is not
> > > "controlled" in any way by me! As laid out in our governance document (
> > > http://gem5.org/Governance#Overview), "gem5 is a meritocratic,
> > > consensus-based community project". Through these emails, and as the
> > chair
> > > of the project management committee, I'm trying to *build* consensus.
> > >
> > > Importantly, there's a reason we're trying to make this push for stable
> > > APIs. I've heard from dozens of current and potential gem5 users that
> > they
> > > want stable gem5 releases. By providing stable releases, we will be
> > > expanding the users of gem5 (and, IMO, making the research and papers
> > that
> > > use gem5 better).
> > >
> > > Could you please clarify the policy on breaking APIs? It makes sense
> for
> > > > releases to maintain stable APIs, but how does that apply to the
> > > > development branch? I'm worried that it will be very hard to make
> > changes
> > > > that don't change any interfaces, and we definitely don't want to
> > > encourage
> > > > a style of development where we just add and add and add without ever
> > > > refactoring or consolidating things. If APIs can continue to change
> as
> > > > needed in the development branch and we just need to warn people
> before
> > > > they're released, that seems reasonable.
> > >
> > >
> > > Let's dig into this deeper. First, I'd like to remind everyone that the
> > > current proposal was the community consensus reached in the gem5
> roadmap
> > > document:
> > >
> > >
> >
> https://docs.google.com/document/d/1fv01HavfkIIqfcgZoKUpojkUKkWujxspRCcvS5cTfkk/edit
> > > .
> > > Of course, we can always change what we decided before :).
> > >
> > > The goal is to provide our users with a stable base to build of off. If
> > we
> > > are constantly changing interfaces, like we do today, it's impossible
> to
> > > build a project based off of gem5. There have been many projects that
> > have
> > > languished 

Re: [gem5-dev] gem5 19.0.0 : New git branching proposal

2019-12-12 Thread Jason Lowe-Power
Hey Abhishek,

Emails will continue to gem5-dev on every changeset pushed to the develop
branch as they do now to master :). We can discuss if we want the same for
feature branches (if we end up using them).

Your first interpretation on gem5 stable is correct (sorry if this wasn't
clear). It will be much more heavily tested than the minute-by-minute
releases from the develop branch. With this testing we will be publishing
the following:
- What (common) workloads are supported (e.g., SPEC, parsec, etc.). Which
workloads we use here will be discussed in the future, stay tuned.
- For all of the workloads, we will publish common statistics for a few
different systems (e.g., time to simulate, IPC, cache miss rates, memory
bandwidth, etc). The systems used and the stats will be discussed in the
future.

The stable release will *not* be a continuous process. The only time the
stable branch will be updated is 1) At releases or 2) if a "major" bug is
encountered. For non-release updates (e.g., for bugs), we'll be very
careful to either re-validate our results or somehow know the results won't
change.

Cheers,
Jason



On Thu, Dec 12, 2019 at 9:38 AM Abhishek Singh <
abhishek.singh199...@gmail.com> wrote:

> Hi Jason,
>
> Thanks for your email. It cleared a lot of misunderstanding which I had.
>
> Is it possible to have an email sent on every commit we make to atleast
> gem5-dev?
> The email list could be different and can be sent to people who are
> interested in this so that it does not spam to gem5-Dev list.
>
> I am talking about gem5 developer branch and not stable.
>
> This is because, it will keep all the interested community members well
> informed about new features that are added, who like to keep note of latest
> changes and merge in their projects as required.
>
> And the way, I understand that a stable gem5 when a user desires is:
> It wants the conference accepted applications (standard workload for
> example spec, parsec, etc to run without any errors on both SE and FS mode
> for every architecture and cpu models.
>
> If that’s what the stable releases are going to be testing before releasing
> it, then a stable release is much more helpful and will have the wide
> reach. But when I read proposal this seems to be step by step process
> reaching it as the final goal. Please correct me on this if I understood it
> wrong.
>
>
> The aim of stable releases is a continuous process and will always be
> doubtful unless tested with all the major conference accepted application
> on every stable releases.
>
> On Thu, Dec 12, 2019 at 12:12 PM Jason Lowe-Power 
> wrote:
>
> > Hi all,
> >
> > First of all, let me say that I hope it's clear that gem5 is not
> > "controlled" in any way by me! As laid out in our governance document (
> > http://gem5.org/Governance#Overview), "gem5 is a meritocratic,
> > consensus-based community project". Through these emails, and as the
> chair
> > of the project management committee, I'm trying to *build* consensus.
> >
> > Importantly, there's a reason we're trying to make this push for stable
> > APIs. I've heard from dozens of current and potential gem5 users that
> they
> > want stable gem5 releases. By providing stable releases, we will be
> > expanding the users of gem5 (and, IMO, making the research and papers
> that
> > use gem5 better).
> >
> > Could you please clarify the policy on breaking APIs? It makes sense for
> > > releases to maintain stable APIs, but how does that apply to the
> > > development branch? I'm worried that it will be very hard to make
> changes
> > > that don't change any interfaces, and we definitely don't want to
> > encourage
> > > a style of development where we just add and add and add without ever
> > > refactoring or consolidating things. If APIs can continue to change as
> > > needed in the development branch and we just need to warn people before
> > > they're released, that seems reasonable.
> >
> >
> > Let's dig into this deeper. First, I'd like to remind everyone that the
> > current proposal was the community consensus reached in the gem5 roadmap
> > document:
> >
> >
> https://docs.google.com/document/d/1fv01HavfkIIqfcgZoKUpojkUKkWujxspRCcvS5cTfkk/edit
> > .
> > Of course, we can always change what we decided before :).
> >
> > The goal is to provide our users with a stable base to build of off. If
> we
> > are constantly changing interfaces, like we do today, it's impossible to
> > build a project based off of gem5. There have been many projects that
> have
> > languished because of this including gem5+SST and dist-gem5. If we had
> well
> > defined interfaces *that weren't constantly changing* I believe it would
> > make gem5 a more widely used project.
> >
> > The proposal is to have *one* API breaking release every year. This was
> > decided based on community feedback in the roadmap document.
> >
> > I (personally) believe we need to slow down and be more deliberate about
> > changing our interfaces. I think that it's a good thing 

Re: [gem5-dev] gem5 19.0.0 : New git branching proposal

2019-12-12 Thread Abhishek Singh
Hi Jason,

Thanks for your email. It cleared a lot of misunderstanding which I had.

Is it possible to have an email sent on every commit we make to atleast
gem5-dev?
The email list could be different and can be sent to people who are
interested in this so that it does not spam to gem5-Dev list.

I am talking about gem5 developer branch and not stable.

This is because, it will keep all the interested community members well
informed about new features that are added, who like to keep note of latest
changes and merge in their projects as required.

And the way, I understand that a stable gem5 when a user desires is:
It wants the conference accepted applications (standard workload for
example spec, parsec, etc to run without any errors on both SE and FS mode
for every architecture and cpu models.

If that’s what the stable releases are going to be testing before releasing
it, then a stable release is much more helpful and will have the wide
reach. But when I read proposal this seems to be step by step process
reaching it as the final goal. Please correct me on this if I understood it
wrong.


The aim of stable releases is a continuous process and will always be
doubtful unless tested with all the major conference accepted application
on every stable releases.

On Thu, Dec 12, 2019 at 12:12 PM Jason Lowe-Power 
wrote:

> Hi all,
>
> First of all, let me say that I hope it's clear that gem5 is not
> "controlled" in any way by me! As laid out in our governance document (
> http://gem5.org/Governance#Overview), "gem5 is a meritocratic,
> consensus-based community project". Through these emails, and as the chair
> of the project management committee, I'm trying to *build* consensus.
>
> Importantly, there's a reason we're trying to make this push for stable
> APIs. I've heard from dozens of current and potential gem5 users that they
> want stable gem5 releases. By providing stable releases, we will be
> expanding the users of gem5 (and, IMO, making the research and papers that
> use gem5 better).
>
> Could you please clarify the policy on breaking APIs? It makes sense for
> > releases to maintain stable APIs, but how does that apply to the
> > development branch? I'm worried that it will be very hard to make changes
> > that don't change any interfaces, and we definitely don't want to
> encourage
> > a style of development where we just add and add and add without ever
> > refactoring or consolidating things. If APIs can continue to change as
> > needed in the development branch and we just need to warn people before
> > they're released, that seems reasonable.
>
>
> Let's dig into this deeper. First, I'd like to remind everyone that the
> current proposal was the community consensus reached in the gem5 roadmap
> document:
>
> https://docs.google.com/document/d/1fv01HavfkIIqfcgZoKUpojkUKkWujxspRCcvS5cTfkk/edit
> .
> Of course, we can always change what we decided before :).
>
> The goal is to provide our users with a stable base to build of off. If we
> are constantly changing interfaces, like we do today, it's impossible to
> build a project based off of gem5. There have been many projects that have
> languished because of this including gem5+SST and dist-gem5. If we had well
> defined interfaces *that weren't constantly changing* I believe it would
> make gem5 a more widely used project.
>
> The proposal is to have *one* API breaking release every year. This was
> decided based on community feedback in the roadmap document.
>
> I (personally) believe we need to slow down and be more deliberate about
> changing our interfaces. I think that it's a good thing that "it will be
> very hard to make changes that don't change any interfaces". This will make
> gem5 more stable and easy for others to use. If you believe you need to
> change an interface, we should have a discussion about it first, and this
> should include a discussion about how the interface change will affect our
> users.
>
> I strongly agree that "we definitely don't want to encourage a style of
> development where we just add and add and add without ever refactoring or
> consolidating things." The current proposal states that when changing an
> interface, we first add a new function and mark the old version as
> deprecated. Then, during the API changing merge window delete all of the
> deprecated functions. This is going to take more maintenance from our side,
> but we now have the resources to do this.
>
> I disagree that "we just need to warn people before [API changes] are
> released" is enough. We need to give them significant (months) of lead
> time, and we need to give them the opportunity to transition. In fact, I
> believe this is what Abhishek was getting at in his follow up.
>
> In general (and not necessarily tied to this proposal), we should try very
> > hard not to have silos where people develop incompatible features and
> don't
> > add to gem5 in a holistic way. It's easiest to wall off functionality
> > behind ifdefs or config 

Re: [gem5-dev] gem5 19.0.0 : New git branching proposal

2019-12-12 Thread Jason Lowe-Power
Hi all,

First of all, let me say that I hope it's clear that gem5 is not
"controlled" in any way by me! As laid out in our governance document (
http://gem5.org/Governance#Overview), "gem5 is a meritocratic,
consensus-based community project". Through these emails, and as the chair
of the project management committee, I'm trying to *build* consensus.

Importantly, there's a reason we're trying to make this push for stable
APIs. I've heard from dozens of current and potential gem5 users that they
want stable gem5 releases. By providing stable releases, we will be
expanding the users of gem5 (and, IMO, making the research and papers that
use gem5 better).

Could you please clarify the policy on breaking APIs? It makes sense for
> releases to maintain stable APIs, but how does that apply to the
> development branch? I'm worried that it will be very hard to make changes
> that don't change any interfaces, and we definitely don't want to encourage
> a style of development where we just add and add and add without ever
> refactoring or consolidating things. If APIs can continue to change as
> needed in the development branch and we just need to warn people before
> they're released, that seems reasonable.


Let's dig into this deeper. First, I'd like to remind everyone that the
current proposal was the community consensus reached in the gem5 roadmap
document:
https://docs.google.com/document/d/1fv01HavfkIIqfcgZoKUpojkUKkWujxspRCcvS5cTfkk/edit.
Of course, we can always change what we decided before :).

The goal is to provide our users with a stable base to build of off. If we
are constantly changing interfaces, like we do today, it's impossible to
build a project based off of gem5. There have been many projects that have
languished because of this including gem5+SST and dist-gem5. If we had well
defined interfaces *that weren't constantly changing* I believe it would
make gem5 a more widely used project.

The proposal is to have *one* API breaking release every year. This was
decided based on community feedback in the roadmap document.

I (personally) believe we need to slow down and be more deliberate about
changing our interfaces. I think that it's a good thing that "it will be
very hard to make changes that don't change any interfaces". This will make
gem5 more stable and easy for others to use. If you believe you need to
change an interface, we should have a discussion about it first, and this
should include a discussion about how the interface change will affect our
users.

I strongly agree that "we definitely don't want to encourage a style of
development where we just add and add and add without ever refactoring or
consolidating things." The current proposal states that when changing an
interface, we first add a new function and mark the old version as
deprecated. Then, during the API changing merge window delete all of the
deprecated functions. This is going to take more maintenance from our side,
but we now have the resources to do this.

I disagree that "we just need to warn people before [API changes] are
released" is enough. We need to give them significant (months) of lead
time, and we need to give them the opportunity to transition. In fact, I
believe this is what Abhishek was getting at in his follow up.

In general (and not necessarily tied to this proposal), we should try very
> hard not to have silos where people develop incompatible features and don't
> add to gem5 in a holistic way. It's easiest to wall off functionality
> behind ifdefs or config options or in separate repositories and then just
> ignore it when implementing new features, but that creates a lot of
> technical debt and complexity which really cripples both gem5 and future
> development and compounds exponentially over time.


I agree with this statement. We need to encourage development in the open
based on the mainline code. I strongly believe the current proposal *makes
this easier*, not harder. Today, many people do exactly what you're
describing. By implementing stable APIs that people can build off of and
allowing branches, we will be making it easier to add features to gem5 in a
holistic way.

Back to the branches, I'm still not a fan. I think it gives people an
> excuse not to make changes ("you want me to refactor 18 months of code for
> that?" (not an actual quote)), on top of other issues. Also there are more
> than two options. You can do your work on top of tree like I do, even when
> adding big new features. It ensures that you don't get too invested in bad
> paths and don't have a mountain of integration work to do when you finally
> decide to bridge back in. It also gives the community an appropriate
> opportunity to provide feedback through review which is not practical when
> dealing with dozens of files and thousands of lines of code with months of
> yet unseen history and iteration.


No one can do what you do, Gabe :D.

More seriously, a couple of points:
- Branches won't be "unseen." Their development 

[gem5-dev] Change in gem5/gem5[master]: mem: Encapsulate mapping gem5 to host address space

2019-12-12 Thread Daniel Carvalho (Gerrit)
Daniel Carvalho has submitted this change. (  
https://gem5-review.googlesource.com/c/public/gem5/+/22610 )


Change subject: mem: Encapsulate mapping gem5 to host address space
..

mem: Encapsulate mapping gem5 to host address space

Create a function to encapsulate mapping an address in gem5's
address space to the host's address space. The returned value can
be used to access the contents of the given address.

As a side effect, make the local variable hostAddr use snake_case
to comply with gem5's coding style.

Change-Id: I2445d3ab4c7ce5746182b307c26cbafc68aa139c
Signed-off-by: Daniel R. Carvalho 
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/22610
Reviewed-by: Nikos Nikoleris 
Maintainer: Nikos Nikoleris 
Tested-by: kokoro 
---
M src/mem/abstract_mem.cc
M src/mem/abstract_mem.hh
2 files changed, 28 insertions(+), 15 deletions(-)

Approvals:
  Nikos Nikoleris: Looks good to me, approved; Looks good to me, approved
  kokoro: Regressions pass



diff --git a/src/mem/abstract_mem.cc b/src/mem/abstract_mem.cc
index d913f05..41b621b 100644
--- a/src/mem/abstract_mem.cc
+++ b/src/mem/abstract_mem.cc
@@ -361,13 +361,13 @@

 assert(pkt->getAddrRange().isSubset(range));

-uint8_t *hostAddr = pmemAddr + pkt->getAddr() - range.start();
+uint8_t *host_addr = toHostAddr(pkt->getAddr());

 if (pkt->cmd == MemCmd::SwapReq) {
 if (pkt->isAtomicOp()) {
 if (pmemAddr) {
-pkt->setData(hostAddr);
-(*(pkt->getAtomicOp()))(hostAddr);
+pkt->setData(host_addr);
+(*(pkt->getAtomicOp()))(host_addr);
 }
 } else {
 std::vector overwrite_val(pkt->getSize());
@@ -381,23 +381,23 @@
 // keep a copy of our possible write value, and copy what is  
at the

 // memory address into the packet
 pkt->writeData(_val[0]);
-pkt->setData(hostAddr);
+pkt->setData(host_addr);

 if (pkt->req->isCondSwap()) {
 if (pkt->getSize() == sizeof(uint64_t)) {
 condition_val64 = pkt->req->getExtraData();
-overwrite_mem = !std::memcmp(_val64,  
hostAddr,
+overwrite_mem = !std::memcmp(_val64,  
host_addr,

  sizeof(uint64_t));
 } else if (pkt->getSize() == sizeof(uint32_t)) {
 condition_val32 = (uint32_t)pkt->req->getExtraData();
-overwrite_mem = !std::memcmp(_val32,  
hostAddr,
+overwrite_mem = !std::memcmp(_val32,  
host_addr,

  sizeof(uint32_t));
 } else
 panic("Invalid size for conditional read/write\n");
 }

 if (overwrite_mem)
-std::memcpy(hostAddr, _val[0], pkt->getSize());
+std::memcpy(host_addr, _val[0], pkt->getSize());

 assert(!pkt->req->isInstFetch());
 TRACE_PACKET("Read/Write");
@@ -412,7 +412,7 @@
 trackLoadLocked(pkt);
 }
 if (pmemAddr) {
-pkt->setData(hostAddr);
+pkt->setData(host_addr);
 }
 TRACE_PACKET(pkt->req->isInstFetch() ? "IFetch" : "Read");
 stats.numReads[pkt->req->masterId()]++;
@@ -428,9 +428,9 @@
 } else if (pkt->isWrite()) {
 if (writeOK(pkt)) {
 if (pmemAddr) {
-pkt->writeData(hostAddr);
-DPRINTF(MemoryAccess, "%s wrote %i bytes to address %x\n",
-__func__, pkt->getSize(), pkt->getAddr());
+pkt->writeData(host_addr);
+DPRINTF(MemoryAccess, "%s write due to %s\n",
+__func__, pkt->print());
 }
 assert(!pkt->req->isInstFetch());
 TRACE_PACKET("Write");
@@ -451,17 +451,17 @@
 {
 assert(pkt->getAddrRange().isSubset(range));

-uint8_t *hostAddr = pmemAddr + pkt->getAddr() - range.start();
+uint8_t *host_addr = toHostAddr(pkt->getAddr());

 if (pkt->isRead()) {
 if (pmemAddr) {
-pkt->setData(hostAddr);
+pkt->setData(host_addr);
 }
 TRACE_PACKET("Read");
 pkt->makeResponse();
 } else if (pkt->isWrite()) {
 if (pmemAddr) {
-pkt->writeData(hostAddr);
+pkt->writeData(host_addr);
 }
 TRACE_PACKET("Write");
 pkt->makeResponse();
@@ -473,7 +473,7 @@
 // through printObj().
 prs->printLabels();
 // Right now we just print the single byte at the specified  
address.

-ccprintf(prs->os, "%s%#x\n", prs->curPrefix(), *hostAddr);
+ccprintf(prs->os, "%s%#x\n", prs->curPrefix(), *host_addr);
 } else {
 panic("AbstractMemory: unimplemented functional command %s",
   

[gem5-dev] Change in gem5/gem5[master]: mem-cache: Move unused prefetches counter update

2019-12-12 Thread Daniel Carvalho (Gerrit)
Daniel Carvalho has submitted this change. (  
https://gem5-review.googlesource.com/c/public/gem5/+/18908 )


Change subject: mem-cache: Move unused prefetches counter update
..

mem-cache: Move unused prefetches counter update

The number of unused prefetches should be updated every time
a block is invalidated, therefore we move the update to within
the corresponding function.

Change-Id: If3ac2ea43611525bd3c36d628d88382042fcb7dc
Signed-off-by: Daniel R. Carvalho 
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/18908
Tested-by: kokoro 
Reviewed-by: Nikos Nikoleris 
Maintainer: Nikos Nikoleris 
---
M src/mem/cache/base.cc
1 file changed, 5 insertions(+), 8 deletions(-)

Approvals:
  Nikos Nikoleris: Looks good to me, approved; Looks good to me, approved
  kokoro: Regressions pass



diff --git a/src/mem/cache/base.cc b/src/mem/cache/base.cc
index ebfb092..d56fcbe 100644
--- a/src/mem/cache/base.cc
+++ b/src/mem/cache/base.cc
@@ -883,9 +883,6 @@
 // Evict valid blocks
 for (const auto& evict_blk : evict_blks) {
 if (evict_blk->isValid()) {
-if (evict_blk->wasPrefetched()) {
-stats.unusedPrefetches++;
-}
 evictBlock(evict_blk, writebacks);
 }
 }
@@ -1461,11 +1458,6 @@
 DPRINTF(CacheRepl, "Evicting %s (%#llx) to make room for "  
\
 "%#llx (%s)\n", blk->print(),  
regenerateBlkAddr(blk),

 addr, is_secure);
-
-if (blk->wasPrefetched()) {
-stats.unusedPrefetches++;
-}
-
 evictBlock(blk, writebacks);
 }
 }
@@ -1489,6 +1481,11 @@
 void
 BaseCache::invalidateBlock(CacheBlk *blk)
 {
+// If block is still marked as prefetched, then it hasn't been used
+if (blk->wasPrefetched()) {
+stats.unusedPrefetches++;
+}
+
 // If handling a block present in the Tags, let it do its invalidation
 // process, which will update stats and invalidate the block itself
 if (blk != tempBlock) {

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/18908
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: master
Gerrit-Change-Id: If3ac2ea43611525bd3c36d628d88382042fcb7dc
Gerrit-Change-Number: 18908
Gerrit-PatchSet: 4
Gerrit-Owner: Daniel Carvalho 
Gerrit-Reviewer: Daniel Carvalho 
Gerrit-Reviewer: Nikos Nikoleris 
Gerrit-Reviewer: kokoro 
Gerrit-CC: Javier Bueno Hedo 
Gerrit-MessageType: merged
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev

[gem5-dev] Change in gem5/gem5[master]: mem-cache: Add print function to ReplaceableEntry

2019-12-12 Thread Daniel Carvalho (Gerrit)
Daniel Carvalho has uploaded this change for review. (  
https://gem5-review.googlesource.com/c/public/gem5/+/23567 )



Change subject: mem-cache: Add print function to ReplaceableEntry
..

mem-cache: Add print function to ReplaceableEntry

Add a basic print function to acquire and display information about
replaceable entries.

Change-Id: I9640113d305fbe086c5bfaf8928a911bfcac50bb
Signed-off-by: Daniel R. Carvalho 
---
M src/mem/cache/cache_blk.hh
M src/mem/cache/replacement_policies/replaceable_entry.hh
2 files changed, 17 insertions(+), 3 deletions(-)



diff --git a/src/mem/cache/cache_blk.hh b/src/mem/cache/cache_blk.hh
index 3bc3786..c3e37e3 100644
--- a/src/mem/cache/cache_blk.hh
+++ b/src/mem/cache/cache_blk.hh
@@ -377,7 +377,8 @@
  *
  * @return string with basic state information
  */
-virtual std::string print() const
+std::string
+print() const override
 {
 /**
  *  state   M   O   E   S   I
@@ -414,9 +415,9 @@
   default:s = 'T'; break; // @TODO add other types
 }
 return csprintf("state: %x (%c) valid: %d writable: %d  
readable: %d "
-"dirty: %d | tag: %#x set: %#x way: %#x", status,  
s,

+"dirty: %d | tag: %#x %s", status, s,
 isValid(), isWritable(), isReadable(), isDirty(),  
tag,

-getSet(), getWay());
+ReplaceableEntry::print());
 }

 /**
diff --git a/src/mem/cache/replacement_policies/replaceable_entry.hh  
b/src/mem/cache/replacement_policies/replaceable_entry.hh

index 2b08749..9a59b1f 100644
--- a/src/mem/cache/replacement_policies/replaceable_entry.hh
+++ b/src/mem/cache/replacement_policies/replaceable_entry.hh
@@ -34,6 +34,8 @@
 #include 
 #include 

+#include "base/cprintf.hh"
+
 /**
  * The replacement data needed by replacement policies. Each replacement  
policy

  * should have its own implementation of replacement data.
@@ -99,6 +101,17 @@
  * @return The way to which this entry belongs.
  */
 uint32_t getWay() const { return _way; }
+
+/**
+ * Prints relevant information about this entry.
+ *
+ * @return A string containg the contents of this entry.
+ */
+virtual std::string
+print() const
+{
+return csprintf("set: %#x way: %#x", getSet(), getWay());
+}
 };

 #endif // __MEM_CACHE_REPLACEMENT_POLICIES_REPLACEABLE_ENTRY_HH_

--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/23567
To unsubscribe, or for help writing mail filters, visit  
https://gem5-review.googlesource.com/settings


Gerrit-Project: public/gem5
Gerrit-Branch: master
Gerrit-Change-Id: I9640113d305fbe086c5bfaf8928a911bfcac50bb
Gerrit-Change-Number: 23567
Gerrit-PatchSet: 1
Gerrit-Owner: Daniel Carvalho 
Gerrit-MessageType: newchange
___
gem5-dev mailing list
gem5-dev@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev