[gem5-dev] Re: Build failed in Jenkins: weekly #99

[Poremba, Matthew via gem5-dev]

[AMD Official Use Only - General]

I don't think I have tried pennant on Vega, but the current status (as of 
several months ago) is:

Two weekly tests have issues with Vega right now:  (1) heterosync with 
sleepMutex and (2) lulesh
Two weekly tests have issues running in full-system: (1) sssp and (2) sssp_ell

We can discuss the details in another thread though.


-Matt

From: Matt Sinclair 
Sent: Monday, January 9, 2023 9:41 AM
To: Poremba, Matthew 
Cc: Bobby Bruce ; Jason Lowe-Power ; 
The gem5 Developer List ; vrama...@wisc.edu
Subject: Re: [gem5-dev] Re: Build failed in Jenkins: weekly #99

Caution: This message originated from an External Source. Use proper caution 
when opening attachments, clicking links, or responding.

Thanks all.  I have validated all weekly and nightly GPU tests pass with my 
changes.

Matt P: we should probably figure out if all tests pass on Vega now, and if so 
move to Vega for nightly and weekly.  My recollection is some workloads 
(Pennant?) were failing with Vega last time we tried?

Matt

On Mon, Jan 9, 2023 at 10:46 AM Poremba, Matthew 
mailto:matthew.pore...@amd.com>> wrote:

[AMD Official Use Only - General]

Thanks all, it looks like everything was taken care of over the weekend.

To answer MattS' question: I've tested weekly on most of my changes and haven't 
found any issues.  I've been primarily making arch-vega changes and the weekly 
tester only tests gcn3 by default though.


-Matt

From: Bobby Bruce mailto:bbr...@ucdavis.edu>>
Sent: Monday, January 9, 2023 7:59 AM
To: The gem5 Developer List mailto:gem5-dev@gem5.org>>
Cc: Jason Lowe-Power mailto:ja...@lowepower.com>>; 
Poremba, Matthew mailto:matthew.pore...@amd.com>>; 
vrama...@wisc.edu<mailto:vrama...@wisc.edu>; Matt Sinclair 
mailto:mattdsinclair.w...@gmail.com>>
Subject: Re: [gem5-dev] Re: Build failed in Jenkins: weekly #99

Caution: This message originated from an External Source. Use proper caution 
when opening attachments, clicking links, or responding.

Thanks Matt!

I've restarted the Weeklys to ensure it's now working. Should be complete over 
the next day or two.

--
Dr. Bobby R. Bruce
Room 3050,
Kemper Hall, UC Davis
Davis,
CA, 95616

web: 
https://www.bobbybruce.net<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.bobbybruce.net%2F=05%7C01%7CMatthew.Poremba%40amd.com%7C29b4a6854f3948accc5908daf268c143%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638088829027721631%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=yEcYPeFe3%2Flgkth9w8Lsi3Lbe5AWMz7qXbXnGnduloo%3D=0>


On Sun, Jan 8, 2023 at 12:48 AM Matt Sinclair via gem5-dev 
mailto:gem5-dev@gem5.org>> wrote:
The chain for the fixes for weekly is here: 
https://gem5-review.googlesource.com/c/public/gem5/+/67199/1<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgem5-review.googlesource.com%2Fc%2Fpublic%2Fgem5%2F%2B%2F67199%2F1=05%7C01%7CMatthew.Poremba%40amd.com%7C29b4a6854f3948accc5908daf268c143%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638088829027721631%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=%2B4mI%2FGKnn6SjFubHxv9tKZ57r8YCpYWMKS%2FL%2FZM%2BPQ0%3D=0>

I have tested that BC gets past the current failure with these 3 fixes 
(previously BC failed in an initialization kernel before the first iteration 
started, so far with the change it completes the first 107/128 iterations ... 
hopefully the last few go smoothly as well).  Obviously I have not tested the 
entire weekly script yet though since that takes multiple days.  I will run 
that in parallel with these being reviewed.

Matt

On Sat, Jan 7, 2023 at 4:12 PM Jason Lowe-Power 
mailto:ja...@lowepower.com>> wrote:
Thanks for quickly digging into this, Matt!

On Sat, Jan 7, 2023 at 1:41 PM Matt Sinclair via gem5-dev 
mailto:gem5-dev@gem5.org>> wrote:
I have confirmed that the Pannotia benchmarks (to my surprise) are using AMD's 
cache bypassing flags for some memory accesses, which Vishnu added support for 
this week.  Good thing the support is added now!  But that is why they are 
failing here -- they hit a corner case Vishnu and I had considered, but 
implemented incorrectly.  I have a fix I am testing now and will push later 
tonight assuming it solves the problem.

Matt

On Fri, Jan 6, 2023 at 10:07 PM Matt Sinclair 
mailto:mattdsinclair.w...@gmail.com>> wrote:
Hi Matt P & Vishnu,

It appears something with the GPU support must have broken between your changes 
this week -- as far as I can tell all of the nightly tests passed when you 
checked in your commits, but something in the more complex benchmarks (BC in 
this case) is breaking:

gem5.opt: build/GCN3_X86/mem/ruby/system/VIPERCoalescer.cc:265: void 
gem5::ruby::VIPERCoalescer::invTCPCallback(gem5::Addr): Assertion 
`m_cache_inv_pkt && m_num_pending_invs > 0' failed.
Vish

[gem5-dev] Re: Build failed in Jenkins: weekly #99

[Matt Sinclair via gem5-dev]

Thanks all.  I have validated all weekly and nightly GPU tests pass with my
changes.

Matt P: we should probably figure out if all tests pass on Vega now, and if
so move to Vega for nightly and weekly.  My recollection is some workloads
(Pennant?) were failing with Vega last time we tried?

Matt

On Mon, Jan 9, 2023 at 10:46 AM Poremba, Matthew 
wrote:

> [AMD Official Use Only - General]
>
>
>
> Thanks all, it looks like everything was taken care of over the weekend.
>
>
>
> To answer MattS’ question: I’ve tested weekly on most of my changes and
> haven’t found any issues.  I’ve been primarily making arch-vega changes and
> the weekly tester only tests gcn3 by default though.
>
>
>
>
>
> -Matt
>
>
>
> *From:* Bobby Bruce 
> *Sent:* Monday, January 9, 2023 7:59 AM
> *To:* The gem5 Developer List 
> *Cc:* Jason Lowe-Power ; Poremba, Matthew <
> matthew.pore...@amd.com>; vrama...@wisc.edu; Matt Sinclair <
> mattdsinclair.w...@gmail.com>
> *Subject:* Re: [gem5-dev] Re: Build failed in Jenkins: weekly #99
>
>
>
> *Caution:* This message originated from an External Source. Use proper
> caution when opening attachments, clicking links, or responding.
>
>
>
> Thanks Matt!
>
>
>
> I've restarted the Weeklys to ensure it's now working. Should be complete
> over the next day or two.
>
>
>
> --
>
> Dr. Bobby R. Bruce
> Room 3050,
> Kemper Hall, UC Davis
> Davis,
> CA, 95616
>
>
>
> web: https://www.bobbybruce.net
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.bobbybruce.net%2F=05%7C01%7CMatthew.Poremba%40amd.com%7Cff4c7fc844b843dc3b6508daf25a8462%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638088767854623756%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=Wxw5ByN%2BTeiOQC4FczRdTg0RA28U5gsznTB9Ax3VofY%3D=0>
>
>
>
>
>
> On Sun, Jan 8, 2023 at 12:48 AM Matt Sinclair via gem5-dev <
> gem5-dev@gem5.org> wrote:
>
> The chain for the fixes for weekly is here:
> https://gem5-review.googlesource.com/c/public/gem5/+/67199/1
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgem5-review.googlesource.com%2Fc%2Fpublic%2Fgem5%2F%2B%2F67199%2F1=05%7C01%7CMatthew.Poremba%40amd.com%7Cff4c7fc844b843dc3b6508daf25a8462%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638088767854623756%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=3%2Fvkw6vJ0TdIsMSo8OrBtZf7oTBa4kO%2F4fz5xQ1ep8Q%3D=0>
>
>
>
> I have tested that BC gets past the current failure with these 3 fixes
> (previously BC failed in an initialization kernel before the first
> iteration started, so far with the change it completes the first 107/128
> iterations ... hopefully the last few go smoothly as well).  Obviously I
> have not tested the entire weekly script yet though since that takes
> multiple days.  I will run that in parallel with these being reviewed.
>
>
>
> Matt
>
>
>
> On Sat, Jan 7, 2023 at 4:12 PM Jason Lowe-Power 
> wrote:
>
> Thanks for quickly digging into this, Matt!
>
>
>
> On Sat, Jan 7, 2023 at 1:41 PM Matt Sinclair via gem5-dev <
> gem5-dev@gem5.org> wrote:
>
> I have confirmed that the Pannotia benchmarks (to my surprise) are using
> AMD's cache bypassing flags for some memory accesses, which Vishnu added
> support for this week.  Good thing the support is added now!  But that is
> why they are failing here -- they hit a corner case Vishnu and I had
> considered, but implemented incorrectly.  I have a fix I am testing now and
> will push later tonight assuming it solves the problem.
>
>
>
> Matt
>
>
>
> On Fri, Jan 6, 2023 at 10:07 PM Matt Sinclair <
> mattdsinclair.w...@gmail.com> wrote:
>
> Hi Matt P & Vishnu,
>
>
>
> It appears something with the GPU support must have broken between your
> changes this week -- as far as I can tell all of the nightly tests passed
> when you checked in your commits, but something in the more complex
> benchmarks (BC in this case) is breaking:
>
> gem5.opt: build/GCN3_X86/mem/ruby/system/VIPERCoalescer.cc:265: void 
> gem5::ruby::VIPERCoalescer::invTCPCallback(gem5::Addr): Assertion 
> `m_cache_inv_pkt && m_num_pending_invs > 0' failed.
>
> Vishnu, did you test your changes with the weekly tests at all?
>
>
>
> Matt P did you test your changes with the weekly tests at all?  And have
> you started bisecting yet to find the offending commit?
>
>
>
> If not, Vishnu I can show you how to do this.  I will be away next week
> (although with intermittent email access) so a fix relying on me may be
> delayed ... but ho

[gem5-dev] Re: Build failed in Jenkins: weekly #99

[Poremba, Matthew via gem5-dev]

[AMD Official Use Only - General]

Thanks all, it looks like everything was taken care of over the weekend.

To answer MattS' question: I've tested weekly on most of my changes and haven't 
found any issues.  I've been primarily making arch-vega changes and the weekly 
tester only tests gcn3 by default though.


-Matt

From: Bobby Bruce 
Sent: Monday, January 9, 2023 7:59 AM
To: The gem5 Developer List 
Cc: Jason Lowe-Power ; Poremba, Matthew 
; vrama...@wisc.edu; Matt Sinclair 

Subject: Re: [gem5-dev] Re: Build failed in Jenkins: weekly #99

Caution: This message originated from an External Source. Use proper caution 
when opening attachments, clicking links, or responding.

Thanks Matt!

I've restarted the Weeklys to ensure it's now working. Should be complete over 
the next day or two.

--
Dr. Bobby R. Bruce
Room 3050,
Kemper Hall, UC Davis
Davis,
CA, 95616

web: 
https://www.bobbybruce.net<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.bobbybruce.net%2F=05%7C01%7CMatthew.Poremba%40amd.com%7Cff4c7fc844b843dc3b6508daf25a8462%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638088767854623756%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=Wxw5ByN%2BTeiOQC4FczRdTg0RA28U5gsznTB9Ax3VofY%3D=0>


On Sun, Jan 8, 2023 at 12:48 AM Matt Sinclair via gem5-dev 
mailto:gem5-dev@gem5.org>> wrote:
The chain for the fixes for weekly is here: 
https://gem5-review.googlesource.com/c/public/gem5/+/67199/1<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgem5-review.googlesource.com%2Fc%2Fpublic%2Fgem5%2F%2B%2F67199%2F1=05%7C01%7CMatthew.Poremba%40amd.com%7Cff4c7fc844b843dc3b6508daf25a8462%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638088767854623756%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=3%2Fvkw6vJ0TdIsMSo8OrBtZf7oTBa4kO%2F4fz5xQ1ep8Q%3D=0>

I have tested that BC gets past the current failure with these 3 fixes 
(previously BC failed in an initialization kernel before the first iteration 
started, so far with the change it completes the first 107/128 iterations ... 
hopefully the last few go smoothly as well).  Obviously I have not tested the 
entire weekly script yet though since that takes multiple days.  I will run 
that in parallel with these being reviewed.

Matt

On Sat, Jan 7, 2023 at 4:12 PM Jason Lowe-Power 
mailto:ja...@lowepower.com>> wrote:
Thanks for quickly digging into this, Matt!

On Sat, Jan 7, 2023 at 1:41 PM Matt Sinclair via gem5-dev 
mailto:gem5-dev@gem5.org>> wrote:
I have confirmed that the Pannotia benchmarks (to my surprise) are using AMD's 
cache bypassing flags for some memory accesses, which Vishnu added support for 
this week.  Good thing the support is added now!  But that is why they are 
failing here -- they hit a corner case Vishnu and I had considered, but 
implemented incorrectly.  I have a fix I am testing now and will push later 
tonight assuming it solves the problem.

Matt

On Fri, Jan 6, 2023 at 10:07 PM Matt Sinclair 
mailto:mattdsinclair.w...@gmail.com>> wrote:
Hi Matt P & Vishnu,

It appears something with the GPU support must have broken between your changes 
this week -- as far as I can tell all of the nightly tests passed when you 
checked in your commits, but something in the more complex benchmarks (BC in 
this case) is breaking:

gem5.opt: build/GCN3_X86/mem/ruby/system/VIPERCoalescer.cc:265: void 
gem5::ruby::VIPERCoalescer::invTCPCallback(gem5::Addr): Assertion 
`m_cache_inv_pkt && m_num_pending_invs > 0' failed.
Vishnu, did you test your changes with the weekly tests at all?

Matt P did you test your changes with the weekly tests at all?  And have you 
started bisecting yet to find the offending commit?

If not, Vishnu I can show you how to do this.  I will be away next week 
(although with intermittent email access) so a fix relying on me may be delayed 
... but hopefully between the three of us we can isolate and figure out which 
commit is causing/fixing.  My intuition says that it's probably one of Vishnu's 
commits, since Matt P's aren't changing the coherence protocol at all, but it's 
not obvious why Vishnu's commits would be affecting the invalidation calls at 
all ...

Thanks,
Matt S.

On Fri, Jan 6, 2023 at 9:54 PM jenkins-no-reply--- via gem5-dev 
mailto:gem5-dev@gem5.org>> wrote:
See 
<https://jenkins.gem5.org/job/weekly/99/display/redirect?page=changes<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjenkins.gem5.org%2Fjob%2Fweekly%2F99%2Fdisplay%2Fredirect%3Fpage%3Dchanges=05%7C01%7CMatthew.Poremba%40amd.com%7Cff4c7fc844b843dc3b6508daf25a8462%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638088767854623756%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=a3chYQiavMeFgmX4qt4HHSo8Vj1q%2FMAAEC6XaIwQqHg%3D=0>>

Changes:

[Bobby R. Bruce] ext: Fix SST Docum

[gem5-dev] Re: Build failed in Jenkins: weekly #99

[Bobby Bruce via gem5-dev]

Thanks Matt!

I've restarted the Weeklys to ensure it's now working. Should be complete
over the next day or two.

--
Dr. Bobby R. Bruce
Room 3050,
Kemper Hall, UC Davis
Davis,
CA, 95616

web: https://www.bobbybruce.net


On Sun, Jan 8, 2023 at 12:48 AM Matt Sinclair via gem5-dev <
gem5-dev@gem5.org> wrote:

> The chain for the fixes for weekly is here:
> https://gem5-review.googlesource.com/c/public/gem5/+/67199/1
>
> I have tested that BC gets past the current failure with these 3 fixes
> (previously BC failed in an initialization kernel before the first
> iteration started, so far with the change it completes the first 107/128
> iterations ... hopefully the last few go smoothly as well).  Obviously I
> have not tested the entire weekly script yet though since that takes
> multiple days.  I will run that in parallel with these being reviewed.
>
> Matt
>
> On Sat, Jan 7, 2023 at 4:12 PM Jason Lowe-Power 
> wrote:
>
>> Thanks for quickly digging into this, Matt!
>>
>> On Sat, Jan 7, 2023 at 1:41 PM Matt Sinclair via gem5-dev <
>> gem5-dev@gem5.org> wrote:
>>
>>> I have confirmed that the Pannotia benchmarks (to my surprise) are using
>>> AMD's cache bypassing flags for some memory accesses, which Vishnu added
>>> support for this week.  Good thing the support is added now!  But that is
>>> why they are failing here -- they hit a corner case Vishnu and I had
>>> considered, but implemented incorrectly.  I have a fix I am testing now and
>>> will push later tonight assuming it solves the problem.
>>>
>>> Matt
>>>
>>> On Fri, Jan 6, 2023 at 10:07 PM Matt Sinclair <
>>> mattdsinclair.w...@gmail.com> wrote:
>>>
 Hi Matt P & Vishnu,

 It appears something with the GPU support must have broken between your
 changes this week -- as far as I can tell all of the nightly tests passed
 when you checked in your commits, but something in the more complex
 benchmarks (BC in this case) is breaking:

 gem5.opt: build/GCN3_X86/mem/ruby/system/VIPERCoalescer.cc:265: void 
 gem5::ruby::VIPERCoalescer::invTCPCallback(gem5::Addr): Assertion 
 `m_cache_inv_pkt && m_num_pending_invs > 0' failed.

 Vishnu, did you test your changes with the weekly tests at all?

 Matt P did you test your changes with the weekly tests at all?  And
 have you started bisecting yet to find the offending commit?

 If not, Vishnu I can show you how to do this.  I will be away next week
 (although with intermittent email access) so a fix relying on me may be
 delayed ... but hopefully between the three of us we can isolate and figure
 out which commit is causing/fixing.  My intuition says that it's probably
 one of Vishnu's commits, since Matt P's aren't changing the coherence
 protocol at all, but it's not obvious why Vishnu's commits would be
 affecting the invalidation calls at all ...

 Thanks,
 Matt S.

 On Fri, Jan 6, 2023 at 9:54 PM jenkins-no-reply--- via gem5-dev <
 gem5-dev@gem5.org> wrote:

> See <
> https://jenkins.gem5.org/job/weekly/99/display/redirect?page=changes>
>
> Changes:
>
> [Bobby R. Bruce] ext: Fix SST Documentation links
>
> [Bobby R. Bruce] tests: Fix the download test
>
> [Bobby R. Bruce] stdlib: Removing incorrect requires.
>
> [Bobby R. Bruce] stdlib: se_binary_workload exits on work items by
> default
>
> [Bobby R. Bruce] configs: Fix unconnected PCI port in SST gem5 config
>
> [Bobby R. Bruce] mem: Add getAddrRanges in HBMCtrl
>
> [Bobby R. Bruce] system-arm: Fix FEAT_PAuth trapping in AArch64
> bootloader
>
> [Bobby R. Bruce] misc: Update version info to v22.0.0.2
>
> [Bobby R. Bruce] misc: Update RELEASE-NOTES.md for v22.0.0.2
>
> [Bobby R. Bruce] stdlib: Fix get_isa_from_str() exception behavior in
> isas.py
>
> [Bobby R. Bruce] dev-amdgpu: Handle ring buffer wrap for PM4 queue
>
> [Bobby R. Bruce] arch-vega: Fix SOPK instruction sign extends
>
> [Bobby R. Bruce] dev-amdgpu: Fix SDMA ring buffer wrap around
>
> [Bobby R. Bruce] arch-x86: X86ISA default vector_string to HygonGenuine
>
> [Bobby R. Bruce] arch-arm: Revert 'Setup TC/ISA at construction time..'
>
> [Bobby R. Bruce] stdlib,configs: Update riscvmatched-fs example
> docstring
>
> [Bobby R. Bruce] configs,stdlib: Fix import in riscvmatched-fs.py
>
> [Bobby R. Bruce] configs,stdlib,tests: Update riscvmatched-fs.py
> to-init
>
> [Bobby R. Bruce] tests: Update riscvmatched tests to use ALL/gem5.opt
>
> [Bobby R. Bruce] configs: Add missing `_pre_instantiate` call in
> "run_lupv.py"
>
> [Bobby R. Bruce] tests: Delete build directory before running KVM in
> nightly
>
> [Bobby R. Bruce] configs: Set CPU vendor to M5 Simulator in apu_se.py
>
> [Bobby R. Bruce] stdlib,python: Allow setting of to tick exits via m5
>
> 

[gem5-dev] Re: Build failed in Jenkins: weekly #99

[Matt Sinclair via gem5-dev]

The chain for the fixes for weekly is here:
https://gem5-review.googlesource.com/c/public/gem5/+/67199/1

I have tested that BC gets past the current failure with these 3 fixes
(previously BC failed in an initialization kernel before the first
iteration started, so far with the change it completes the first 107/128
iterations ... hopefully the last few go smoothly as well).  Obviously I
have not tested the entire weekly script yet though since that takes
multiple days.  I will run that in parallel with these being reviewed.

Matt

On Sat, Jan 7, 2023 at 4:12 PM Jason Lowe-Power  wrote:

> Thanks for quickly digging into this, Matt!
>
> On Sat, Jan 7, 2023 at 1:41 PM Matt Sinclair via gem5-dev <
> gem5-dev@gem5.org> wrote:
>
>> I have confirmed that the Pannotia benchmarks (to my surprise) are using
>> AMD's cache bypassing flags for some memory accesses, which Vishnu added
>> support for this week.  Good thing the support is added now!  But that is
>> why they are failing here -- they hit a corner case Vishnu and I had
>> considered, but implemented incorrectly.  I have a fix I am testing now and
>> will push later tonight assuming it solves the problem.
>>
>> Matt
>>
>> On Fri, Jan 6, 2023 at 10:07 PM Matt Sinclair <
>> mattdsinclair.w...@gmail.com> wrote:
>>
>>> Hi Matt P & Vishnu,
>>>
>>> It appears something with the GPU support must have broken between your
>>> changes this week -- as far as I can tell all of the nightly tests passed
>>> when you checked in your commits, but something in the more complex
>>> benchmarks (BC in this case) is breaking:
>>>
>>> gem5.opt: build/GCN3_X86/mem/ruby/system/VIPERCoalescer.cc:265: void 
>>> gem5::ruby::VIPERCoalescer::invTCPCallback(gem5::Addr): Assertion 
>>> `m_cache_inv_pkt && m_num_pending_invs > 0' failed.
>>>
>>> Vishnu, did you test your changes with the weekly tests at all?
>>>
>>> Matt P did you test your changes with the weekly tests at all?  And have
>>> you started bisecting yet to find the offending commit?
>>>
>>> If not, Vishnu I can show you how to do this.  I will be away next week
>>> (although with intermittent email access) so a fix relying on me may be
>>> delayed ... but hopefully between the three of us we can isolate and figure
>>> out which commit is causing/fixing.  My intuition says that it's probably
>>> one of Vishnu's commits, since Matt P's aren't changing the coherence
>>> protocol at all, but it's not obvious why Vishnu's commits would be
>>> affecting the invalidation calls at all ...
>>>
>>> Thanks,
>>> Matt S.
>>>
>>> On Fri, Jan 6, 2023 at 9:54 PM jenkins-no-reply--- via gem5-dev <
>>> gem5-dev@gem5.org> wrote:
>>>
 See <
 https://jenkins.gem5.org/job/weekly/99/display/redirect?page=changes>

 Changes:

 [Bobby R. Bruce] ext: Fix SST Documentation links

 [Bobby R. Bruce] tests: Fix the download test

 [Bobby R. Bruce] stdlib: Removing incorrect requires.

 [Bobby R. Bruce] stdlib: se_binary_workload exits on work items by
 default

 [Bobby R. Bruce] configs: Fix unconnected PCI port in SST gem5 config

 [Bobby R. Bruce] mem: Add getAddrRanges in HBMCtrl

 [Bobby R. Bruce] system-arm: Fix FEAT_PAuth trapping in AArch64
 bootloader

 [Bobby R. Bruce] misc: Update version info to v22.0.0.2

 [Bobby R. Bruce] misc: Update RELEASE-NOTES.md for v22.0.0.2

 [Bobby R. Bruce] stdlib: Fix get_isa_from_str() exception behavior in
 isas.py

 [Bobby R. Bruce] dev-amdgpu: Handle ring buffer wrap for PM4 queue

 [Bobby R. Bruce] arch-vega: Fix SOPK instruction sign extends

 [Bobby R. Bruce] dev-amdgpu: Fix SDMA ring buffer wrap around

 [Bobby R. Bruce] arch-x86: X86ISA default vector_string to HygonGenuine

 [Bobby R. Bruce] arch-arm: Revert 'Setup TC/ISA at construction time..'

 [Bobby R. Bruce] stdlib,configs: Update riscvmatched-fs example
 docstring

 [Bobby R. Bruce] configs,stdlib: Fix import in riscvmatched-fs.py

 [Bobby R. Bruce] configs,stdlib,tests: Update riscvmatched-fs.py to-init

 [Bobby R. Bruce] tests: Update riscvmatched tests to use ALL/gem5.opt

 [Bobby R. Bruce] configs: Add missing `_pre_instantiate` call in
 "run_lupv.py"

 [Bobby R. Bruce] tests: Delete build directory before running KVM in
 nightly

 [Bobby R. Bruce] configs: Set CPU vendor to M5 Simulator in apu_se.py

 [Bobby R. Bruce] stdlib,python: Allow setting of to tick exits via m5

 [Bobby R. Bruce] stdlib, configs: Updating configs/example/gem5_library

 [Bobby R. Bruce] arch-arm: Setup TC/ISA at construction time 2nd attempt

 [Bobby R. Bruce] scons: Remove -Werror for the gem5 v22.1 release

 [Bobby R. Bruce] base: Update the version to v22.1.0.0

 [Bobby R. Bruce] python,tests: Update Resource URL path to v22-1

 [Bobby R. Bruce] stdlib: Update the gem5 resources' version to 

[gem5-dev] Re: Build failed in Jenkins: weekly #99

[Jason Lowe-Power via gem5-dev]

Thanks for quickly digging into this, Matt!

On Sat, Jan 7, 2023 at 1:41 PM Matt Sinclair via gem5-dev 
wrote:

> I have confirmed that the Pannotia benchmarks (to my surprise) are using
> AMD's cache bypassing flags for some memory accesses, which Vishnu added
> support for this week.  Good thing the support is added now!  But that is
> why they are failing here -- they hit a corner case Vishnu and I had
> considered, but implemented incorrectly.  I have a fix I am testing now and
> will push later tonight assuming it solves the problem.
>
> Matt
>
> On Fri, Jan 6, 2023 at 10:07 PM Matt Sinclair <
> mattdsinclair.w...@gmail.com> wrote:
>
>> Hi Matt P & Vishnu,
>>
>> It appears something with the GPU support must have broken between your
>> changes this week -- as far as I can tell all of the nightly tests passed
>> when you checked in your commits, but something in the more complex
>> benchmarks (BC in this case) is breaking:
>>
>> gem5.opt: build/GCN3_X86/mem/ruby/system/VIPERCoalescer.cc:265: void 
>> gem5::ruby::VIPERCoalescer::invTCPCallback(gem5::Addr): Assertion 
>> `m_cache_inv_pkt && m_num_pending_invs > 0' failed.
>>
>> Vishnu, did you test your changes with the weekly tests at all?
>>
>> Matt P did you test your changes with the weekly tests at all?  And have
>> you started bisecting yet to find the offending commit?
>>
>> If not, Vishnu I can show you how to do this.  I will be away next week
>> (although with intermittent email access) so a fix relying on me may be
>> delayed ... but hopefully between the three of us we can isolate and figure
>> out which commit is causing/fixing.  My intuition says that it's probably
>> one of Vishnu's commits, since Matt P's aren't changing the coherence
>> protocol at all, but it's not obvious why Vishnu's commits would be
>> affecting the invalidation calls at all ...
>>
>> Thanks,
>> Matt S.
>>
>> On Fri, Jan 6, 2023 at 9:54 PM jenkins-no-reply--- via gem5-dev <
>> gem5-dev@gem5.org> wrote:
>>
>>> See <
>>> https://jenkins.gem5.org/job/weekly/99/display/redirect?page=changes>
>>>
>>> Changes:
>>>
>>> [Bobby R. Bruce] ext: Fix SST Documentation links
>>>
>>> [Bobby R. Bruce] tests: Fix the download test
>>>
>>> [Bobby R. Bruce] stdlib: Removing incorrect requires.
>>>
>>> [Bobby R. Bruce] stdlib: se_binary_workload exits on work items by
>>> default
>>>
>>> [Bobby R. Bruce] configs: Fix unconnected PCI port in SST gem5 config
>>>
>>> [Bobby R. Bruce] mem: Add getAddrRanges in HBMCtrl
>>>
>>> [Bobby R. Bruce] system-arm: Fix FEAT_PAuth trapping in AArch64
>>> bootloader
>>>
>>> [Bobby R. Bruce] misc: Update version info to v22.0.0.2
>>>
>>> [Bobby R. Bruce] misc: Update RELEASE-NOTES.md for v22.0.0.2
>>>
>>> [Bobby R. Bruce] stdlib: Fix get_isa_from_str() exception behavior in
>>> isas.py
>>>
>>> [Bobby R. Bruce] dev-amdgpu: Handle ring buffer wrap for PM4 queue
>>>
>>> [Bobby R. Bruce] arch-vega: Fix SOPK instruction sign extends
>>>
>>> [Bobby R. Bruce] dev-amdgpu: Fix SDMA ring buffer wrap around
>>>
>>> [Bobby R. Bruce] arch-x86: X86ISA default vector_string to HygonGenuine
>>>
>>> [Bobby R. Bruce] arch-arm: Revert 'Setup TC/ISA at construction time..'
>>>
>>> [Bobby R. Bruce] stdlib,configs: Update riscvmatched-fs example docstring
>>>
>>> [Bobby R. Bruce] configs,stdlib: Fix import in riscvmatched-fs.py
>>>
>>> [Bobby R. Bruce] configs,stdlib,tests: Update riscvmatched-fs.py to-init
>>>
>>> [Bobby R. Bruce] tests: Update riscvmatched tests to use ALL/gem5.opt
>>>
>>> [Bobby R. Bruce] configs: Add missing `_pre_instantiate` call in
>>> "run_lupv.py"
>>>
>>> [Bobby R. Bruce] tests: Delete build directory before running KVM in
>>> nightly
>>>
>>> [Bobby R. Bruce] configs: Set CPU vendor to M5 Simulator in apu_se.py
>>>
>>> [Bobby R. Bruce] stdlib,python: Allow setting of to tick exits via m5
>>>
>>> [Bobby R. Bruce] stdlib, configs: Updating configs/example/gem5_library
>>>
>>> [Bobby R. Bruce] arch-arm: Setup TC/ISA at construction time 2nd attempt
>>>
>>> [Bobby R. Bruce] scons: Remove -Werror for the gem5 v22.1 release
>>>
>>> [Bobby R. Bruce] base: Update the version to v22.1.0.0
>>>
>>> [Bobby R. Bruce] python,tests: Update Resource URL path to v22-1
>>>
>>> [Bobby R. Bruce] stdlib: Update the gem5 resources' version to "v22.1"
>>>
>>> [Bobby R. Bruce] util-docker: Update gcn-gpu Docker to use v22-1 ROCM
>>> patch
>>>
>>> [Bobby R. Bruce] util-docker: Add v22-1 tag to docker-compose.yaml
>>>
>>> [Bobby R. Bruce] tests: Update the compiler-tests.sh to use the v22-1
>>> images
>>>
>>> [Bobby R. Bruce] tests: Abstract the docker image tag for Nightly tests
>>>
>>> [Bobby R. Bruce] tests: Update nightly test docker image tags to v22-1
>>>
>>> [Bobby R. Bruce] tests: Abstract the docker image tag for Weekly tests
>>>
>>> [Bobby R. Bruce] tests: Update weekly test docker image tags to v22-1
>>>
>>> [Bobby R. Bruce] util-gem5art: Fix incorrect type of size in
>>> `createArtifact`
>>>
>>> [Bobby R. Bruce] tests: Update presubmit.sh to use v22-1 docker 

[gem5-dev] Re: Build failed in Jenkins: weekly #99

[Matt Sinclair via gem5-dev]

I have confirmed that the Pannotia benchmarks (to my surprise) are using
AMD's cache bypassing flags for some memory accesses, which Vishnu added
support for this week.  Good thing the support is added now!  But that is
why they are failing here -- they hit a corner case Vishnu and I had
considered, but implemented incorrectly.  I have a fix I am testing now and
will push later tonight assuming it solves the problem.

Matt

On Fri, Jan 6, 2023 at 10:07 PM Matt Sinclair 
wrote:

> Hi Matt P & Vishnu,
>
> It appears something with the GPU support must have broken between your
> changes this week -- as far as I can tell all of the nightly tests passed
> when you checked in your commits, but something in the more complex
> benchmarks (BC in this case) is breaking:
>
> gem5.opt: build/GCN3_X86/mem/ruby/system/VIPERCoalescer.cc:265: void 
> gem5::ruby::VIPERCoalescer::invTCPCallback(gem5::Addr): Assertion 
> `m_cache_inv_pkt && m_num_pending_invs > 0' failed.
>
> Vishnu, did you test your changes with the weekly tests at all?
>
> Matt P did you test your changes with the weekly tests at all?  And have
> you started bisecting yet to find the offending commit?
>
> If not, Vishnu I can show you how to do this.  I will be away next week
> (although with intermittent email access) so a fix relying on me may be
> delayed ... but hopefully between the three of us we can isolate and figure
> out which commit is causing/fixing.  My intuition says that it's probably
> one of Vishnu's commits, since Matt P's aren't changing the coherence
> protocol at all, but it's not obvious why Vishnu's commits would be
> affecting the invalidation calls at all ...
>
> Thanks,
> Matt S.
>
> On Fri, Jan 6, 2023 at 9:54 PM jenkins-no-reply--- via gem5-dev <
> gem5-dev@gem5.org> wrote:
>
>> See > >
>>
>> Changes:
>>
>> [Bobby R. Bruce] ext: Fix SST Documentation links
>>
>> [Bobby R. Bruce] tests: Fix the download test
>>
>> [Bobby R. Bruce] stdlib: Removing incorrect requires.
>>
>> [Bobby R. Bruce] stdlib: se_binary_workload exits on work items by default
>>
>> [Bobby R. Bruce] configs: Fix unconnected PCI port in SST gem5 config
>>
>> [Bobby R. Bruce] mem: Add getAddrRanges in HBMCtrl
>>
>> [Bobby R. Bruce] system-arm: Fix FEAT_PAuth trapping in AArch64 bootloader
>>
>> [Bobby R. Bruce] misc: Update version info to v22.0.0.2
>>
>> [Bobby R. Bruce] misc: Update RELEASE-NOTES.md for v22.0.0.2
>>
>> [Bobby R. Bruce] stdlib: Fix get_isa_from_str() exception behavior in
>> isas.py
>>
>> [Bobby R. Bruce] dev-amdgpu: Handle ring buffer wrap for PM4 queue
>>
>> [Bobby R. Bruce] arch-vega: Fix SOPK instruction sign extends
>>
>> [Bobby R. Bruce] dev-amdgpu: Fix SDMA ring buffer wrap around
>>
>> [Bobby R. Bruce] arch-x86: X86ISA default vector_string to HygonGenuine
>>
>> [Bobby R. Bruce] arch-arm: Revert 'Setup TC/ISA at construction time..'
>>
>> [Bobby R. Bruce] stdlib,configs: Update riscvmatched-fs example docstring
>>
>> [Bobby R. Bruce] configs,stdlib: Fix import in riscvmatched-fs.py
>>
>> [Bobby R. Bruce] configs,stdlib,tests: Update riscvmatched-fs.py to-init
>>
>> [Bobby R. Bruce] tests: Update riscvmatched tests to use ALL/gem5.opt
>>
>> [Bobby R. Bruce] configs: Add missing `_pre_instantiate` call in
>> "run_lupv.py"
>>
>> [Bobby R. Bruce] tests: Delete build directory before running KVM in
>> nightly
>>
>> [Bobby R. Bruce] configs: Set CPU vendor to M5 Simulator in apu_se.py
>>
>> [Bobby R. Bruce] stdlib,python: Allow setting of to tick exits via m5
>>
>> [Bobby R. Bruce] stdlib, configs: Updating configs/example/gem5_library
>>
>> [Bobby R. Bruce] arch-arm: Setup TC/ISA at construction time 2nd attempt
>>
>> [Bobby R. Bruce] scons: Remove -Werror for the gem5 v22.1 release
>>
>> [Bobby R. Bruce] base: Update the version to v22.1.0.0
>>
>> [Bobby R. Bruce] python,tests: Update Resource URL path to v22-1
>>
>> [Bobby R. Bruce] stdlib: Update the gem5 resources' version to "v22.1"
>>
>> [Bobby R. Bruce] util-docker: Update gcn-gpu Docker to use v22-1 ROCM
>> patch
>>
>> [Bobby R. Bruce] util-docker: Add v22-1 tag to docker-compose.yaml
>>
>> [Bobby R. Bruce] tests: Update the compiler-tests.sh to use the v22-1
>> images
>>
>> [Bobby R. Bruce] tests: Abstract the docker image tag for Nightly tests
>>
>> [Bobby R. Bruce] tests: Update nightly test docker image tags to v22-1
>>
>> [Bobby R. Bruce] tests: Abstract the docker image tag for Weekly tests
>>
>> [Bobby R. Bruce] tests: Update weekly test docker image tags to v22-1
>>
>> [Bobby R. Bruce] util-gem5art: Fix incorrect type of size in
>> `createArtifact`
>>
>> [Bobby R. Bruce] tests: Update presubmit.sh to use v22-1 docker images
>>
>> [Bobby R. Bruce] ext: Update ext/sst/README.md for v22.1 release
>>
>> [Bobby R. Bruce] python: Remove 'scheduleTickExit' in favor of
>> 'exitSimLoop'
>>
>> [Bobby R. Bruce] configs: Fix x86-gapbs-benchmarks.py example
>>
>> [Bobby R. Bruce] configs: Alter x86-npb-benchmarks.py 

[gem5-dev] Re: Build failed in Jenkins: weekly #99

[Matt Sinclair via gem5-dev]

Hi Matt P & Vishnu,

It appears something with the GPU support must have broken between your
changes this week -- as far as I can tell all of the nightly tests passed
when you checked in your commits, but something in the more complex
benchmarks (BC in this case) is breaking:

gem5.opt: build/GCN3_X86/mem/ruby/system/VIPERCoalescer.cc:265: void
gem5::ruby::VIPERCoalescer::invTCPCallback(gem5::Addr): Assertion
`m_cache_inv_pkt && m_num_pending_invs > 0' failed.

Vishnu, did you test your changes with the weekly tests at all?

Matt P did you test your changes with the weekly tests at all?  And have
you started bisecting yet to find the offending commit?

If not, Vishnu I can show you how to do this.  I will be away next week
(although with intermittent email access) so a fix relying on me may be
delayed ... but hopefully between the three of us we can isolate and figure
out which commit is causing/fixing.  My intuition says that it's probably
one of Vishnu's commits, since Matt P's aren't changing the coherence
protocol at all, but it's not obvious why Vishnu's commits would be
affecting the invalidation calls at all ...

Thanks,
Matt S.

On Fri, Jan 6, 2023 at 9:54 PM jenkins-no-reply--- via gem5-dev <
gem5-dev@gem5.org> wrote:

> See 
>
> Changes:
>
> [Bobby R. Bruce] ext: Fix SST Documentation links
>
> [Bobby R. Bruce] tests: Fix the download test
>
> [Bobby R. Bruce] stdlib: Removing incorrect requires.
>
> [Bobby R. Bruce] stdlib: se_binary_workload exits on work items by default
>
> [Bobby R. Bruce] configs: Fix unconnected PCI port in SST gem5 config
>
> [Bobby R. Bruce] mem: Add getAddrRanges in HBMCtrl
>
> [Bobby R. Bruce] system-arm: Fix FEAT_PAuth trapping in AArch64 bootloader
>
> [Bobby R. Bruce] misc: Update version info to v22.0.0.2
>
> [Bobby R. Bruce] misc: Update RELEASE-NOTES.md for v22.0.0.2
>
> [Bobby R. Bruce] stdlib: Fix get_isa_from_str() exception behavior in
> isas.py
>
> [Bobby R. Bruce] dev-amdgpu: Handle ring buffer wrap for PM4 queue
>
> [Bobby R. Bruce] arch-vega: Fix SOPK instruction sign extends
>
> [Bobby R. Bruce] dev-amdgpu: Fix SDMA ring buffer wrap around
>
> [Bobby R. Bruce] arch-x86: X86ISA default vector_string to HygonGenuine
>
> [Bobby R. Bruce] arch-arm: Revert 'Setup TC/ISA at construction time..'
>
> [Bobby R. Bruce] stdlib,configs: Update riscvmatched-fs example docstring
>
> [Bobby R. Bruce] configs,stdlib: Fix import in riscvmatched-fs.py
>
> [Bobby R. Bruce] configs,stdlib,tests: Update riscvmatched-fs.py to-init
>
> [Bobby R. Bruce] tests: Update riscvmatched tests to use ALL/gem5.opt
>
> [Bobby R. Bruce] configs: Add missing `_pre_instantiate` call in
> "run_lupv.py"
>
> [Bobby R. Bruce] tests: Delete build directory before running KVM in
> nightly
>
> [Bobby R. Bruce] configs: Set CPU vendor to M5 Simulator in apu_se.py
>
> [Bobby R. Bruce] stdlib,python: Allow setting of to tick exits via m5
>
> [Bobby R. Bruce] stdlib, configs: Updating configs/example/gem5_library
>
> [Bobby R. Bruce] arch-arm: Setup TC/ISA at construction time 2nd attempt
>
> [Bobby R. Bruce] scons: Remove -Werror for the gem5 v22.1 release
>
> [Bobby R. Bruce] base: Update the version to v22.1.0.0
>
> [Bobby R. Bruce] python,tests: Update Resource URL path to v22-1
>
> [Bobby R. Bruce] stdlib: Update the gem5 resources' version to "v22.1"
>
> [Bobby R. Bruce] util-docker: Update gcn-gpu Docker to use v22-1 ROCM patch
>
> [Bobby R. Bruce] util-docker: Add v22-1 tag to docker-compose.yaml
>
> [Bobby R. Bruce] tests: Update the compiler-tests.sh to use the v22-1
> images
>
> [Bobby R. Bruce] tests: Abstract the docker image tag for Nightly tests
>
> [Bobby R. Bruce] tests: Update nightly test docker image tags to v22-1
>
> [Bobby R. Bruce] tests: Abstract the docker image tag for Weekly tests
>
> [Bobby R. Bruce] tests: Update weekly test docker image tags to v22-1
>
> [Bobby R. Bruce] util-gem5art: Fix incorrect type of size in
> `createArtifact`
>
> [Bobby R. Bruce] tests: Update presubmit.sh to use v22-1 docker images
>
> [Bobby R. Bruce] ext: Update ext/sst/README.md for v22.1 release
>
> [Bobby R. Bruce] python: Remove 'scheduleTickExit' in favor of
> 'exitSimLoop'
>
> [Bobby R. Bruce] configs: Fix x86-gapbs-benchmarks.py example
>
> [Bobby R. Bruce] configs: Alter x86-npb-benchmarks.py to exit after WORKEND
>
> [Bobby R. Bruce] misc: Update .mailmap
>
> [Bobby R. Bruce] tests: Remove get_runtime_isa() from parsec_disk_run.py
>
> [Bobby R. Bruce] misc: Update RELEASE-NOTES.md for v22.1.0.0
>
> [rogerycchang] arch-riscv: add RV32 ADFIMU_Zfh instruction tests
>
> [rtatiefo] base: Remove unused output.hh dependency from trace.cc
>
> [vramadas] gpu-compute,mem-ruby: Add support for GPU cache bypassing
>
> [Bobby R. Bruce] scons: Re-add -Werror for gem5 develop branch
>
> [Bobby R. Bruce] misc: Update version info for develop branch
>
> [matthew.poremba] arch-vega: Fix signed BFE instructions
>
> [matthew.poremba]