[gem5-users] Re: Running the PARSEC benchmark with RAM > 3GB
Also, I'm getting the following error on running the command: build/X86/gem5.opt configs/example/gem5_library/x86-parsec-benchmarks.py --benchmark vips --size simsmall Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(3,0) CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.19.83 #1 Hardware name: , BIOS 06/08/2008 Call Trace: dump_stack+0x5d/0x79 panic+0xe2/0x236 mount_block_root+0x2b0/0x2e4 ? set_debug_rodata+0xc/0xc prepare_namespace+0x15b/0x191 kernel_init_freeable+0x23c/0x24c ? rest_init+0xa0/0xa0 kernel_init+0x5/0xf0 ret_from_fork+0x35/0x40 Kernel Offset: disabled ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(3,0) ]--- Any help with how to resolve this would be appreciated. I used gem5-resources to build the disk image for the benchmark and followed the steps mentioned in https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/parsec/. I only changed the vm_memory to 16384 and cores to 8 in parsec.json. On Wed, Jun 14, 2023 at 11:57 AM Vincent Abraham wrote: > Hi everyone, > Is it possible in any way to run the provided x86-parsec-benchmarks.py > script for a system with >3GB of memory? Is there any other way around this > like running the fs.py script for parsec benchmarks? > > Thanks, > Vincent > ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Re: Question about checkpoints
Hi, I've realised though I'm not sure this may work as intended. Do the > checkpoints gem5 generates include a memory dump of instructions loaded > from a binary as well as stack/heap working data too? I'm intending for all > instructions to be loaded out of the binary I pass on checkpoint restore, > rather than out of the checkpoint. Is anybody able to confirm for me > whether or not this is actually the case? I doubt that restoring will work the way you intend to. Based on my understanding, the checkpoint contains a dump of the physical memory and the architectural state and the simulation will restore from exactly the same state/memory image. That means the simulation will resume with the old program loaded into the memory. -Ayaz On Wed, Jun 14, 2023 at 4:09 AM muke101 via gem5-users wrote: > I'm generating checkpoints from spec for a research project. I have a use > case where I'm introducing novel instructions but they only have OoO core > semantics rather than program semantics, and any binary I modify to include > them has the exact same addresses for functions, basic blocks etc as an > unmodified binary. > > My method to generate checkpoints has been to run a binary without novel > instructions natively to generate simpoints, generate the checkpoints with > gem5 passing the unmodified binaries, then restore the checkpoints passing > the modified binaries that have the novel instructions. > > I've realised though I'm not sure this may work as intended. Do the > checkpoints gem5 generates include a memory dump of instructions loaded > from a binary as well as stack/heap working data too? I'm intending for all > instructions to be loaded out of the binary I pass on checkpoint restore, > rather than out of the checkpoint. Is anybody able to confirm for me > whether or not this is actually the case? > > Thanks. > > > Sent from Proton Mail mobile > > > ___ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org > ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Re: Writing a script to run multiple simulations at once
Hi Jason, it seems like this is exactly what I was looking for. Awesome to hear that things will get more user friendly in next versions. Best Derek Am 14.06.23 um 20:05 schrieb Jason Lowe-Power via gem5-users: Hi all, You can use python multiprocessing with gem5. See https://github.com/gem5/gem5/tree/develop/src/python/gem5/utils/multiprocessing for details. So, in theory, you can create scripts such that you can run `./gem5 run.py` which will run a set of different experiments. You could potentially even send a subset of stats back to the main `run.py` script for post-processing, though I haven't tested that, yet. We are planning to add extensions to the simulator module to make this more user friendly in gem5 23.1 (the release after gem5 23.0). Cheers, Jason On Wed, Jun 14, 2023 at 11:01 AM Eliot Moss via gem5-users wrote: On 6/14/2023 11:30 AM, Derek Christ wrote: > Hello Eliot Moss, [one ell please] > a shared Python file with parameter settings sounds useful. > What I meant with running gem5 without the gem5 executable was to use the > compiled library directly from the Python configuration script. > From what I have seen, the gem5 executable sets up some internal state and > then directly calls the embedded Python interpreter to launch the > user-provided script. > But as I see it there is no technical reason why it shouldn't be possible to > call this setup routine directly from Python. This would reduce the > complexity to only one single Python script. Well, there may remain value to having a standard setup/run script that invokes a user supplied script. It helps keep gem5 per se separate from the user's setup / configuration - a principle of modularity. A quick look at main.cc suggests you may be right that this *could* be done, though I have no idea what those various setup functions do and whether any of that would be hard to do from python. What I suppose I am missing is the motivation - why such a change would be substantially better. My applications tend to be quite complex and I find I need the layers of script, for various reasons. Maybe this has more to do with preference to write in python vs bash scripts vs C++ code. gem5 is not currently packaged as a library, I don't think, though I suppose it could be. Given the amount of existing projects and infrastructure, one would need to continue to support the current way of doing things as well. This might further complicate the system and its maintenance - one hopes by not very much. HTH - EM ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org ___ gem5-users mailing list --gem5-users@gem5.org To unsubscribe send an email togem5-users-le...@gem5.org___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Re: Writing a script to run multiple simulations at once
Hi all, You can use python multiprocessing with gem5. See https://github.com/gem5/gem5/tree/develop/src/python/gem5/utils/multiprocessing for details. So, in theory, you can create scripts such that you can run `./gem5 run.py` which will run a set of different experiments. You could potentially even send a subset of stats back to the main `run.py` script for post-processing, though I haven't tested that, yet. We are planning to add extensions to the simulator module to make this more user friendly in gem5 23.1 (the release after gem5 23.0). Cheers, Jason On Wed, Jun 14, 2023 at 11:01 AM Eliot Moss via gem5-users < gem5-users@gem5.org> wrote: > On 6/14/2023 11:30 AM, Derek Christ wrote: > > Hello Eliot Moss, [one ell please] > > > a shared Python file with parameter settings sounds useful. > > > What I meant with running gem5 without the gem5 executable was to use the > > compiled library directly from the Python configuration script. > > > From what I have seen, the gem5 executable sets up some internal state > and > > then directly calls the embedded Python interpreter to launch the > > user-provided script. > > > But as I see it there is no technical reason why it shouldn't be > possible to > > call this setup routine directly from Python. This would reduce the > > complexity to only one single Python script. > > Well, there may remain value to having a standard setup/run script that > invokes a user supplied script. It helps keep gem5 per se separate from > the > user's setup / configuration - a principle of modularity. > > A quick look at main.cc suggests you may be right that this *could* be > done, > though I have no idea what those various setup functions do and whether > any of > that would be hard to do from python. What I suppose I am missing is the > motivation - why such a change would be substantially better. My > applications > tend to be quite complex and I find I need the layers of script, for > various > reasons. Maybe this has more to do with preference to write in python vs > bash > scripts vs C++ code. > > gem5 is not currently packaged as a library, I don't think, though I > suppose > it could be. Given the amount of existing projects and infrastructure, one > would need to continue to support the current way of doing things as well. > This might further complicate the system and its maintenance - one hopes by > not very much. > > HTH - EM > ___ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org > ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Re: Writing a script to run multiple simulations at once
On 6/14/2023 11:30 AM, Derek Christ wrote: Hello Eliot Moss, [one ell please] a shared Python file with parameter settings sounds useful. What I meant with running gem5 without the gem5 executable was to use the compiled library directly from the Python configuration script. From what I have seen, the gem5 executable sets up some internal state and then directly calls the embedded Python interpreter to launch the user-provided script. But as I see it there is no technical reason why it shouldn't be possible to call this setup routine directly from Python. This would reduce the complexity to only one single Python script. Well, there may remain value to having a standard setup/run script that invokes a user supplied script. It helps keep gem5 per se separate from the user's setup / configuration - a principle of modularity. A quick look at main.cc suggests you may be right that this *could* be done, though I have no idea what those various setup functions do and whether any of that would be hard to do from python. What I suppose I am missing is the motivation - why such a change would be substantially better. My applications tend to be quite complex and I find I need the layers of script, for various reasons. Maybe this has more to do with preference to write in python vs bash scripts vs C++ code. gem5 is not currently packaged as a library, I don't think, though I suppose it could be. Given the amount of existing projects and infrastructure, one would need to continue to support the current way of doing things as well. This might further complicate the system and its maintenance - one hopes by not very much. HTH - EM ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Running the PARSEC benchmark with RAM > 3GB
Hi everyone, Is it possible in any way to run the provided x86-parsec-benchmarks.py script for a system with >3GB of memory? Is there any other way around this like running the fs.py script for parsec benchmarks? Thanks, Vincent ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Re: Writing a script to run multiple simulations at once
Hello Elliot Moss, a shared Python file with parameter settings sounds useful. What I meant with running gem5 without the gem5 executable was to use the compiled library directly from the Python configuration script. From what I have seen, the gem5 executable sets up some internal state and then directly calls the embedded Python interpreter to launch the user-provided script. But as I see it there is no technical reason why it shouldn't be possible to call this setup routine directly from Python. This would reduce the complexity to only one single Python script. Best Derek Am 14.06.23 um 14:38 schrieb Eliot Moss via gem5-users: On 6/14/2023 2:32 AM, Derek Christ via gem5-users wrote: Hello, maybe I have missed something in the official docs, but I'm not sure how to run multiple simulations with different parameters concurrently to speed up the process. What I have done is I created a Python script that sets environment variables and then kicks-off gem5 which in turn runs another Python script that reads those environment variables to configure the simulation. But I'm sure there has to be a better way? I think it might be easier if it would be possible to run gem5 purely with Python (without the gem5 executable) because then it would be easy to pass custom parameters to the gem5 configuration. Is there something that I missed? Thanks Best Derek I use scripts that setup environment variables, command line parameters, and a python file with python parameter settings. The latter is placed in a directory unique to the run. It happens that my scripts are shell scripts, but with varying degrees of pain, other scripting languages, or even hard coding in C or something, would work. I just use what I am skilled with. You *must* have the gem4 executable, however. The bulk of the simulation is run with that (optimizing compiled) C++ code. It implements the actual modules as well as the event driven simulation part. Python is used mostly for configuring the simulation before the event loop is kicked off. If the simulator were written entirely in python it would be much slower. HTH - Eliot Moss ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Re: Writing a script to run multiple simulations at once
On 6/14/2023 2:32 AM, Derek Christ via gem5-users wrote: Hello, maybe I have missed something in the official docs, but I'm not sure how to run multiple simulations with different parameters concurrently to speed up the process. What I have done is I created a Python script that sets environment variables and then kicks-off gem5 which in turn runs another Python script that reads those environment variables to configure the simulation. But I'm sure there has to be a better way? I think it might be easier if it would be possible to run gem5 purely with Python (without the gem5 executable) because then it would be easy to pass custom parameters to the gem5 configuration. Is there something that I missed? Thanks Best Derek I use scripts that setup environment variables, command line parameters, and a python file with python parameter settings. The latter is placed in a directory unique to the run. It happens that my scripts are shell scripts, but with varying degrees of pain, other scripting languages, or even hard coding in C or something, would work. I just use what I am skilled with. You *must* have the gem4 executable, however. The bulk of the simulation is run with that (optimizing compiled) C++ code. It implements the actual modules as well as the event driven simulation part. Python is used mostly for configuring the simulation before the event loop is kicked off. If the simulator were written entirely in python it would be much slower. HTH - Eliot Moss ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Re: Writing a script to run multiple simulations at once
Hi Ayaz, thanks for your quick response. Yeah, argparse is one possibility. But it is cumbersome and bugprone to implement a new option for every single configuration parameter I want to vary. Also, I would need a way not only to parse the options, but also to generate the gem5 command with the parameters on-the-fly. As far as I know, argparse does not provide this functionality. So another option that I thought of is to serialize my custom configuration class, pass it to gem5, and deserialize it again in the gem5 configuration script. Best Derek Am 14.06.23 um 09:21 schrieb Ayaz Akram via gem5-users: Hi Derek, I am not sure if I have understood your question correctly, but the gem5 interface is basically a Python run script in which you can add parameters as well (that can be set from the command line or in shell scripts). For example, you can look at the following link to see how `argparse` is used for this purpose: https://www.gem5.org/documentation/learning_gem5/part1/cache_config/ -Ayaz On Tue, Jun 13, 2023 at 11:32 PM Derek Christ via gem5-users wrote: Hello, maybe I have missed something in the official docs, but I'm not sure how to run multiple simulations with different parameters concurrently to speed up the process. What I have done is I created a Python script that sets environment variables and then kicks-off gem5 which in turn runs another Python script that reads those environment variables to configure the simulation. But I'm sure there has to be a better way? I think it might be easier if it would be possible to run gem5 purely with Python (without the gem5 executable) because then it would be easy to pass custom parameters to the gem5 configuration. Is there something that I missed? Thanks Best Derek ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org ___ gem5-users mailing list --gem5-users@gem5.org To unsubscribe send an email togem5-users-le...@gem5.org___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Question about checkpoints
I'm generating checkpoints from spec for a research project. I have a use case where I'm introducing novel instructions but they only have OoO core semantics rather than program semantics, and any binary I modify to include them has the exact same addresses for functions, basic blocks etc as an unmodified binary. My method to generate checkpoints has been to run a binary without novel instructions natively to generate simpoints, generate the checkpoints with gem5 passing the unmodified binaries, then restore the checkpoints passing the modified binaries that have the novel instructions. I've realised though I'm not sure this may work as intended. Do the checkpoints gem5 generates include a memory dump of instructions loaded from a binary as well as stack/heap working data too? I'm intending for all instructions to be loaded out of the binary I pass on checkpoint restore, rather than out of the checkpoint. Is anybody able to confirm for me whether or not this is actually the case? Thanks. Sent from Proton Mail mobile___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Re: Writing a script to run multiple simulations at once
Hi Derek, I am not sure if I have understood your question correctly, but the gem5 interface is basically a Python run script in which you can add parameters as well (that can be set from the command line or in shell scripts). For example, you can look at the following link to see how `argparse` is used for this purpose: https://www.gem5.org/documentation/learning_gem5/part1/cache_config/ -Ayaz On Tue, Jun 13, 2023 at 11:32 PM Derek Christ via gem5-users < gem5-users@gem5.org> wrote: > Hello, > > maybe I have missed something in the official docs, but I'm not sure how > to run multiple simulations with different parameters concurrently to > speed up the process. > > What I have done is I created a Python script that sets environment > variables and then kicks-off gem5 which in turn runs another Python > script that reads those environment variables to configure the simulation. > But I'm sure there has to be a better way? > > I think it might be easier if it would be possible to run gem5 purely > with Python (without the gem5 executable) because then it would be easy > to pass custom parameters to the gem5 configuration. > > Is there something that I missed? > > Thanks > > Best > Derek > ___ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org > ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Writing a script to run multiple simulations at once
Hello, maybe I have missed something in the official docs, but I'm not sure how to run multiple simulations with different parameters concurrently to speed up the process. What I have done is I created a Python script that sets environment variables and then kicks-off gem5 which in turn runs another Python script that reads those environment variables to configure the simulation. But I'm sure there has to be a better way? I think it might be easier if it would be possible to run gem5 purely with Python (without the gem5 executable) because then it would be easy to pass custom parameters to the gem5 configuration. Is there something that I missed? Thanks Best Derek ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
