Re: [gem5-users] How to collect stats from my benchmark on FS mode
I don't see how this is possible, unless you are taking further checkpoints from "m5 readfile". We'd have to understand why there are so many. On Thu, Oct 31, 2019 at 7:10 PM Iago . wrote: > > Using this script : m5 checkpoint; m5 readfile | sh , I left the computer > running the simulation and when I came back there were not one but 5 > checkpoints created at the m5 folder, on the terminal I found this: > Writing checkpoint > info: Entering event queue @ 1181764407936. Starting simulation... > info: Entering event queue @ 1183993091106. Starting simulation... > Writing checkpoint > info: Entering event queue @ 1183993091820. Starting simulation... > Writing checkpoint > info: Entering event queue @ 1186122452592. Starting simulation... > Writing checkpoint > info: Entering event queue @ 1188503609748. Starting simulation... > warn: ClockedObject system.cpu2: More than one power state change request > encountered within the same simulation tick 1189294336902 > warn: ClockedObject system.cpu0: More than one power state change request > encountered within the same simulation tick 1189300250250 > Writing checkpoint > Exiting @ tick 1189651167684 because checkpoint > > Is there any of the checkpoints better suited for simulation or can I use any > of them. > > De: Ciro Santilli > Enviado: quinta-feira, 24 de outubro de 2019 03:48 > Para: Iago . > Cc: gem5 users mailing list > Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode > > On Wed, Oct 23, 2019 at 11:18 PM Iago . wrote: > > > > Also I forgot to ask on the last message but could you specify which init > > script youre talking about and where i would put m5 checkpoint; m5 readfile > > | sh . The reason I worry is because testing from checkpoints made with > > hack_back_ckpt the results were very insconsistent running my control > > precompiled binary.I tried a couple of times and the results varied from > > 0.01 to 0.0025 simulated seconds with different checkpoints of course, > > where as with the manual checkpoints the results were more simillar(still > > different of course since i was manually executing the script through m5 > > readfile |sh) only changing from 0.0025 to around 0.0026. I do agree that > > making these tests reproducible is very important. > > Thanks for the help again > > "m5 checkpoint; m5 readfile | sh" goes inside the file pointed to by > --script, explained a bit further at: > https://stackoverflow.com/questions/58523380/how-to-get-stats-for-several-gem5-full-system-userland-benchmark-programs-under/58523381#58523381 > > > ____________ > > De: Ciro Santilli > > Enviado: quarta-feira, 23 de outubro de 2019 17:32 > > Para: Iago . > > Cc: gem5 users mailing list > > Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode > > > > On Wed, Oct 23, 2019 at 7:37 PM Iago . wrote: > > > > > > Sorry to ask but is there any reason to do it in the init script, isnt > > > the results of the control algorithm performance benchmark going to be > > > affected by the system still booting up? > > > > You'd have to study your system to understand if the init starting > > separate threads or not, and if you need that initialization or not. > > > > To be able to better make sense of the benchmark, I would craft an > > init that does not launch any userland processes, or else the stats > > will be split across programs. > > > > The Buildroot system I linked to, does not launch any processes, > > Ubuntu likely spaws something, but we can just make it use our own > > minimal init which does minimal initialization like mounting /proc > > etc. and runs the benchmark witihout spawning anything else. > > > > Running automatically without manual intervention is preferred to make > > things reproducible. > > > > > Also, i have sucessfully created a checkpoint on the simulation right > > > after the terminal shows root@genericarmv8:~# and am restoring to it and > > > executing the script through the command m5 readfile | sh , but when i > > > check the stats.txt there are two sets of simulation statistics, am I > > > correct to assume that the first one is from whenever I commanded the > > > dumpstats and the last one from whenever the simulation actually ends? > > > > Correct. > > > > > Thanks for the help anyway > > > > > > De: Ciro Santilli > > > Enviado: quarta-feira, 23 de outubro de 2019 09:35 > > &g
Re: [gem5-users] How to collect stats from my benchmark on FS mode
Using this script : m5 checkpoint; m5 readfile | sh , I left the computer running the simulation and when I came back there were not one but 5 checkpoints created at the m5 folder, on the terminal I found this: Writing checkpoint info: Entering event queue @ 1181764407936. Starting simulation... info: Entering event queue @ 1183993091106. Starting simulation... Writing checkpoint info: Entering event queue @ 1183993091820. Starting simulation... Writing checkpoint info: Entering event queue @ 1186122452592. Starting simulation... Writing checkpoint info: Entering event queue @ 1188503609748. Starting simulation... warn: ClockedObject system.cpu2: More than one power state change request encountered within the same simulation tick 1189294336902 warn: ClockedObject system.cpu0: More than one power state change request encountered within the same simulation tick 1189300250250 Writing checkpoint Exiting @ tick 1189651167684 because checkpoint Is there any of the checkpoints better suited for simulation or can I use any of them. De: Ciro Santilli Enviado: quinta-feira, 24 de outubro de 2019 03:48 Para: Iago . Cc: gem5 users mailing list Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode On Wed, Oct 23, 2019 at 11:18 PM Iago . wrote: > > Also I forgot to ask on the last message but could you specify which init > script youre talking about and where i would put m5 checkpoint; m5 readfile | > sh . The reason I worry is because testing from checkpoints made with > hack_back_ckpt the results were very insconsistent running my control > precompiled binary.I tried a couple of times and the results varied from 0.01 > to 0.0025 simulated seconds with different checkpoints of course, where as > with the manual checkpoints the results were more simillar(still different of > course since i was manually executing the script through m5 readfile |sh) > only changing from 0.0025 to around 0.0026. I do agree that making these > tests reproducible is very important. > Thanks for the help again "m5 checkpoint; m5 readfile | sh" goes inside the file pointed to by --script, explained a bit further at: https://stackoverflow.com/questions/58523380/how-to-get-stats-for-several-gem5-full-system-userland-benchmark-programs-under/58523381#58523381 > > De: Ciro Santilli > Enviado: quarta-feira, 23 de outubro de 2019 17:32 > Para: Iago . > Cc: gem5 users mailing list > Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode > > On Wed, Oct 23, 2019 at 7:37 PM Iago . wrote: > > > > Sorry to ask but is there any reason to do it in the init script, isnt the > > results of the control algorithm performance benchmark going to be affected > > by the system still booting up? > > You'd have to study your system to understand if the init starting > separate threads or not, and if you need that initialization or not. > > To be able to better make sense of the benchmark, I would craft an > init that does not launch any userland processes, or else the stats > will be split across programs. > > The Buildroot system I linked to, does not launch any processes, > Ubuntu likely spaws something, but we can just make it use our own > minimal init which does minimal initialization like mounting /proc > etc. and runs the benchmark witihout spawning anything else. > > Running automatically without manual intervention is preferred to make > things reproducible. > > > Also, i have sucessfully created a checkpoint on the simulation right after > > the terminal shows root@genericarmv8:~# and am restoring to it and > > executing the script through the command m5 readfile | sh , but when i > > check the stats.txt there are two sets of simulation statistics, am I > > correct to assume that the first one is from whenever I commanded the > > dumpstats and the last one from whenever the simulation actually ends? > > Correct. > > > Thanks for the help anyway > > ____________ > > De: Ciro Santilli > > Enviado: quarta-feira, 23 de outubro de 2019 09:35 > > Para: gem5 users mailing list ; > > iagosilvestr...@hotmail.com > > Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode > > > > In summary: > > > > - run your benchmark from the init executable > > - to run a single benchmark with different parameters without > > rebooting, do in your init script: "m5 checkpoint; m5 readfile | sh". > > This is what the "configs/boot/hack_back_ckpt.rcS" but I think that > > script is overly complicated. > > - to modify the executable without having to reboot, attach a second > > disk imag
Re: [gem5-users] How to collect stats from my benchmark on FS mode
On Wed, Oct 23, 2019 at 11:18 PM Iago . wrote: > > Also I forgot to ask on the last message but could you specify which init > script youre talking about and where i would put m5 checkpoint; m5 readfile | > sh . The reason I worry is because testing from checkpoints made with > hack_back_ckpt the results were very insconsistent running my control > precompiled binary.I tried a couple of times and the results varied from 0.01 > to 0.0025 simulated seconds with different checkpoints of course, where as > with the manual checkpoints the results were more simillar(still different of > course since i was manually executing the script through m5 readfile |sh) > only changing from 0.0025 to around 0.0026. I do agree that making these > tests reproducible is very important. > Thanks for the help again "m5 checkpoint; m5 readfile | sh" goes inside the file pointed to by --script, explained a bit further at: https://stackoverflow.com/questions/58523380/how-to-get-stats-for-several-gem5-full-system-userland-benchmark-programs-under/58523381#58523381 > > De: Ciro Santilli > Enviado: quarta-feira, 23 de outubro de 2019 17:32 > Para: Iago . > Cc: gem5 users mailing list > Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode > > On Wed, Oct 23, 2019 at 7:37 PM Iago . wrote: > > > > Sorry to ask but is there any reason to do it in the init script, isnt the > > results of the control algorithm performance benchmark going to be affected > > by the system still booting up? > > You'd have to study your system to understand if the init starting > separate threads or not, and if you need that initialization or not. > > To be able to better make sense of the benchmark, I would craft an > init that does not launch any userland processes, or else the stats > will be split across programs. > > The Buildroot system I linked to, does not launch any processes, > Ubuntu likely spaws something, but we can just make it use our own > minimal init which does minimal initialization like mounting /proc > etc. and runs the benchmark witihout spawning anything else. > > Running automatically without manual intervention is preferred to make > things reproducible. > > > Also, i have sucessfully created a checkpoint on the simulation right after > > the terminal shows root@genericarmv8:~# and am restoring to it and > > executing the script through the command m5 readfile | sh , but when i > > check the stats.txt there are two sets of simulation statistics, am I > > correct to assume that the first one is from whenever I commanded the > > dumpstats and the last one from whenever the simulation actually ends? > > Correct. > > > Thanks for the help anyway > > ____________ > > De: Ciro Santilli > > Enviado: quarta-feira, 23 de outubro de 2019 09:35 > > Para: gem5 users mailing list ; > > iagosilvestr...@hotmail.com > > Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode > > > > In summary: > > > > - run your benchmark from the init executable > > - to run a single benchmark with different parameters without > > rebooting, do in your init script: "m5 checkpoint; m5 readfile | sh". > > This is what the "configs/boot/hack_back_ckpt.rcS" but I think that > > script is overly complicated. > > - to modify the executable without having to reboot, attach a second > > disk image: > > https://stackoverflow.com/questions/50862906/how-to-attach-multiple-disk-images-in-a-simulation-with-gem5-fs-py > > - to only count only benchmark instructions, do "m5 resetstats && > > ./run-benchmark && m5 dumpstats". If that is not precise enough, > > modify the source of your benchmark with m5ops instructions that do > > resetstats and dumpstats > > > > Here's a fully automated Buildroot setup that helps with most of that: > > https://github.com/cirosantilli/linux-kernel-module-cheat/tree/d1f16390d0cd53a337e0014ce627803b1d6282de#gem5-run-benchmark > > > > On Tue, Oct 22, 2019 at 11:52 PM Iago . wrote: > > > > > > I have been trying to use gem5 to analyze the performance of different > > > control algorithms of an UAV. Using the full system mode simulation for > > > that with an ARM image I would like to know if theres a way to write a > > > script where I run my cross-compiled binary already moved to the system > > > image and collect the stats only from the time it took to run the control > > > binary. > > > ___ > > > gem5-users mailing list > > > gem5-users@gem5.org > > > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users ___ gem5-users mailing list gem5-users@gem5.org http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Re: [gem5-users] How to collect stats from my benchmark on FS mode
Also I forgot to ask on the last message but could you specify which init script youre talking about and where i would put m5 checkpoint; m5 readfile | sh . The reason I worry is because testing from checkpoints made with hack_back_ckpt the results were very insconsistent running my control precompiled binary.I tried a couple of times and the results varied from 0.01 to 0.0025 simulated seconds with different checkpoints of course, where as with the manual checkpoints the results were more simillar(still different of course since i was manually executing the script through m5 readfile |sh) only changing from 0.0025 to around 0.0026. I do agree that making these tests reproducible is very important. Thanks for the help again De: Ciro Santilli Enviado: quarta-feira, 23 de outubro de 2019 17:32 Para: Iago . Cc: gem5 users mailing list Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode On Wed, Oct 23, 2019 at 7:37 PM Iago . wrote: > > Sorry to ask but is there any reason to do it in the init script, isnt the > results of the control algorithm performance benchmark going to be affected > by the system still booting up? You'd have to study your system to understand if the init starting separate threads or not, and if you need that initialization or not. To be able to better make sense of the benchmark, I would craft an init that does not launch any userland processes, or else the stats will be split across programs. The Buildroot system I linked to, does not launch any processes, Ubuntu likely spaws something, but we can just make it use our own minimal init which does minimal initialization like mounting /proc etc. and runs the benchmark witihout spawning anything else. Running automatically without manual intervention is preferred to make things reproducible. > Also, i have sucessfully created a checkpoint on the simulation right after > the terminal shows root@genericarmv8:~# and am restoring to it and executing > the script through the command m5 readfile | sh , but when i check the > stats.txt there are two sets of simulation statistics, am I correct to assume > that the first one is from whenever I commanded the dumpstats and the last > one from whenever the simulation actually ends? Correct. > Thanks for the help anyway > > De: Ciro Santilli > Enviado: quarta-feira, 23 de outubro de 2019 09:35 > Para: gem5 users mailing list ; > iagosilvestr...@hotmail.com > Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode > > In summary: > > - run your benchmark from the init executable > - to run a single benchmark with different parameters without > rebooting, do in your init script: "m5 checkpoint; m5 readfile | sh". > This is what the "configs/boot/hack_back_ckpt.rcS" but I think that > script is overly complicated. > - to modify the executable without having to reboot, attach a second > disk image: > https://stackoverflow.com/questions/50862906/how-to-attach-multiple-disk-images-in-a-simulation-with-gem5-fs-py > - to only count only benchmark instructions, do "m5 resetstats && > ./run-benchmark && m5 dumpstats". If that is not precise enough, > modify the source of your benchmark with m5ops instructions that do > resetstats and dumpstats > > Here's a fully automated Buildroot setup that helps with most of that: > https://github.com/cirosantilli/linux-kernel-module-cheat/tree/d1f16390d0cd53a337e0014ce627803b1d6282de#gem5-run-benchmark > > On Tue, Oct 22, 2019 at 11:52 PM Iago . wrote: > > > > I have been trying to use gem5 to analyze the performance of different > > control algorithms of an UAV. Using the full system mode simulation for > > that with an ARM image I would like to know if theres a way to write a > > script where I run my cross-compiled binary already moved to the system > > image and collect the stats only from the time it took to run the control > > binary. > > ___ > > gem5-users mailing list > > gem5-users@gem5.org > > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users ___ gem5-users mailing list gem5-users@gem5.org http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Re: [gem5-users] How to collect stats from my benchmark on FS mode
On Wed, Oct 23, 2019 at 7:37 PM Iago . wrote: > > Sorry to ask but is there any reason to do it in the init script, isnt the > results of the control algorithm performance benchmark going to be affected > by the system still booting up? You'd have to study your system to understand if the init starting separate threads or not, and if you need that initialization or not. To be able to better make sense of the benchmark, I would craft an init that does not launch any userland processes, or else the stats will be split across programs. The Buildroot system I linked to, does not launch any processes, Ubuntu likely spaws something, but we can just make it use our own minimal init which does minimal initialization like mounting /proc etc. and runs the benchmark witihout spawning anything else. Running automatically without manual intervention is preferred to make things reproducible. > Also, i have sucessfully created a checkpoint on the simulation right after > the terminal shows root@genericarmv8:~# and am restoring to it and executing > the script through the command m5 readfile | sh , but when i check the > stats.txt there are two sets of simulation statistics, am I correct to assume > that the first one is from whenever I commanded the dumpstats and the last > one from whenever the simulation actually ends? Correct. > Thanks for the help anyway > > De: Ciro Santilli > Enviado: quarta-feira, 23 de outubro de 2019 09:35 > Para: gem5 users mailing list ; > iagosilvestr...@hotmail.com > Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode > > In summary: > > - run your benchmark from the init executable > - to run a single benchmark with different parameters without > rebooting, do in your init script: "m5 checkpoint; m5 readfile | sh". > This is what the "configs/boot/hack_back_ckpt.rcS" but I think that > script is overly complicated. > - to modify the executable without having to reboot, attach a second > disk image: > https://stackoverflow.com/questions/50862906/how-to-attach-multiple-disk-images-in-a-simulation-with-gem5-fs-py > - to only count only benchmark instructions, do "m5 resetstats && > ./run-benchmark && m5 dumpstats". If that is not precise enough, > modify the source of your benchmark with m5ops instructions that do > resetstats and dumpstats > > Here's a fully automated Buildroot setup that helps with most of that: > https://github.com/cirosantilli/linux-kernel-module-cheat/tree/d1f16390d0cd53a337e0014ce627803b1d6282de#gem5-run-benchmark > > On Tue, Oct 22, 2019 at 11:52 PM Iago . wrote: > > > > I have been trying to use gem5 to analyze the performance of different > > control algorithms of an UAV. Using the full system mode simulation for > > that with an ARM image I would like to know if theres a way to write a > > script where I run my cross-compiled binary already moved to the system > > image and collect the stats only from the time it took to run the control > > binary. > > ___ > > gem5-users mailing list > > gem5-users@gem5.org > > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users ___ gem5-users mailing list gem5-users@gem5.org http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Re: [gem5-users] How to collect stats from my benchmark on FS mode
Sorry to ask but is there any reason to do it in the init script, isnt the results of the control algorithm performance benchmark going to be affected by the system still booting up? Also, i have sucessfully created a checkpoint on the simulation right after the terminal shows root@genericarmv8:~# and am restoring to it and executing the script through the command m5 readfile | sh , but when i check the stats.txt there are two sets of simulation statistics, am I correct to assume that the first one is from whenever I commanded the dumpstats and the last one from whenever the simulation actually ends? Thanks for the help anyway De: Ciro Santilli Enviado: quarta-feira, 23 de outubro de 2019 09:35 Para: gem5 users mailing list ; iagosilvestr...@hotmail.com Assunto: Re: [gem5-users] How to collect stats from my benchmark on FS mode In summary: - run your benchmark from the init executable - to run a single benchmark with different parameters without rebooting, do in your init script: "m5 checkpoint; m5 readfile | sh". This is what the "configs/boot/hack_back_ckpt.rcS" but I think that script is overly complicated. - to modify the executable without having to reboot, attach a second disk image: https://stackoverflow.com/questions/50862906/how-to-attach-multiple-disk-images-in-a-simulation-with-gem5-fs-py - to only count only benchmark instructions, do "m5 resetstats && ./run-benchmark && m5 dumpstats". If that is not precise enough, modify the source of your benchmark with m5ops instructions that do resetstats and dumpstats Here's a fully automated Buildroot setup that helps with most of that: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/d1f16390d0cd53a337e0014ce627803b1d6282de#gem5-run-benchmark On Tue, Oct 22, 2019 at 11:52 PM Iago . wrote: > > I have been trying to use gem5 to analyze the performance of different > control algorithms of an UAV. Using the full system mode simulation for that > with an ARM image I would like to know if theres a way to write a script > where I run my cross-compiled binary already moved to the system image and > collect the stats only from the time it took to run the control binary. > ___ > gem5-users mailing list > gem5-users@gem5.org > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users ___ gem5-users mailing list gem5-users@gem5.org http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
Re: [gem5-users] How to collect stats from my benchmark on FS mode
In summary: - run your benchmark from the init executable - to run a single benchmark with different parameters without rebooting, do in your init script: "m5 checkpoint; m5 readfile | sh". This is what the "configs/boot/hack_back_ckpt.rcS" but I think that script is overly complicated. - to modify the executable without having to reboot, attach a second disk image: https://stackoverflow.com/questions/50862906/how-to-attach-multiple-disk-images-in-a-simulation-with-gem5-fs-py - to only count only benchmark instructions, do "m5 resetstats && ./run-benchmark && m5 dumpstats". If that is not precise enough, modify the source of your benchmark with m5ops instructions that do resetstats and dumpstats Here's a fully automated Buildroot setup that helps with most of that: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/d1f16390d0cd53a337e0014ce627803b1d6282de#gem5-run-benchmark On Tue, Oct 22, 2019 at 11:52 PM Iago . wrote: > > I have been trying to use gem5 to analyze the performance of different > control algorithms of an UAV. Using the full system mode simulation for that > with an ARM image I would like to know if theres a way to write a script > where I run my cross-compiled binary already moved to the system image and > collect the stats only from the time it took to run the control binary. > ___ > gem5-users mailing list > gem5-users@gem5.org > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users ___ gem5-users mailing list gem5-users@gem5.org http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
