Re: [gem5-users] Script to convert Gem5 output to McPAT

[dipu . 7009]

Dear Andreas,

Thanks for the reply.
I checked that SimpleCPU is now broken into three other CPU model. Hence, I 
tried the timingsimplecpu. Well, can we try to write script for the 
timingsimplecpu. It is important for benchmark applications. I will also check 
that can I run the simulation with the SimpleCPU only. 

Thanks and best regards,

Faisal

> On Feb 17, 2018, at 4:08 AM, Andreas Brokalakis  wrote:
> 
> Hi Faisal,
> 
> I'll check the errors and come to you. However I should mention that you 
> should not use the AtomicSimple xml template with the config that you are 
> using. You are trying to simulate a far more complex CPU and therefore its 
> xml description is quite different.
> 
> Best,
> Andreas
> 
>> On Fri, Feb 16, 2018 at 8:02 AM, F. A. Faisal  wrote:
>> Dear Mr. Andreas,
>> 
>> Thanks a lot for your scripts on mcpat.
>> However, I had tried to convert my stats file and config file to a new xml 
>> file using the GEM5ToMcPAT.py script.
>> Where, I found that the initial mcpat xml file remained unchanged as the 
>> output file even I used the converting script file of GEM5ToMcPAT.py and 
>> even no error had been shown other than the lots of warnings.
>> Please let me know, if I am missing some thing.
>> 
>> ./GEM5ToMcPAT.py ../../moriya/gem5/m5out/stats.txt 
>> ../../moriya/gem5/m5out/config.json 
>> ../mcpat/ProcessorDescriptionFiles/Alpha21364.xml -o Mesh256MG4GB.xml
>> ./GEM5ToMcPAT.py ../../moriya/gem5/m5out/stats.txt 
>> ../../moriya/gem5/m5out/config.json 
>> ../mcpat/ProcessorDescriptionFiles/Xeon.xml -o Mesh256MG4GB.xml
>> 
>> build/X86_MESI_Two_Level/gem5.opt configs/example/se.py -c 
>> ../nas/NPB3.3.1/NPB3.3-OMP/bin/mg.S.x_256 -n256 --ruby 
>> --cpu-type=TimingSimpleCPU --caches --l2cache --l2_size=16MB --num-l2=256 
>> --l1d_size=64kB --l1i_size=64kB --l1d_assoc=2 --l1i_assoc=2 
>> --network=garnet2.0 --mem-size=4GB  --num-dirs=256 --topology=Mesh_XY 
>> --mesh-rows=16 --options=256 --cpu-clock=1GHz
>> 
>> Plus, when I used the x86_AtomicSimpleCPU_template.xml, it provides me the 
>> below errors-
>> 
>> Traceback (most recent call last):
>>   File "./GEM5ToMcPAT.py", line 181, in 
>> main()
>>   File "./GEM5ToMcPAT.py", line 53, in main
>> dumpMcpatOut(opts.out)
>>   File "./GEM5ToMcPAT.py", line 80, in dumpMcpatOut
>> exprs[i] = str(eval(exprs[i]))
>>   File "", line 1
>> {u'name': None
>> ^
>> SyntaxError: unexpected EOF while parsing
>> 
>> NB- I also can share my stats file and json file, if you want in your 
>> private e-mail.
>> 
>> Thanks a lot in advance.
>> 
>> Yours faithfully,
>> 
>> F.A. Faisal
>> 
>> School of Information Science, JAIST
>> Member of Inoguchi Lab
>> Mob: 080-3048-7558
>> 
>>> On Fri, Feb 16, 2018 at 2:59 PM, F. A. Faisal  wrote:
>>> Dear Mr. Andreas,
>>> 
>>> Thanks a lot for your scripts on mcpat.
>>> However, I had tried to convert my stats file and config file to a new xml 
>>> file using the GEM5ToMcPAT.py script.
>>> Where, I found that the initial mcpat xml file remained unchanged as the 
>>> output file even I used the converting script file of GEM5ToMcPAT.py and 
>>> even no error had been shown other than the lots of warnings.
>>> Please let me know, if I am missing some thing.
>>> 
>>> ./GEM5ToMcPAT.py ../../moriya/gem5/m5out/stats.txt 
>>> ../../moriya/gem5/m5out/config.json 
>>> ../mcpat/ProcessorDescriptionFiles/Alpha21364.xml -o Mesh256MG4GB.xml
>>> ./GEM5ToMcPAT.py ../../moriya/gem5/m5out/stats.txt 
>>> ../../moriya/gem5/m5out/config.json 
>>> ../mcpat/ProcessorDescriptionFiles/Xeon.xml -o Mesh256MG4GB.xml
>>> 
>>> build/X86_MESI_Two_Level/gem5.opt configs/example/se.py -c 
>>> ../nas/NPB3.3.1/NPB3.3-OMP/bin/mg.S.x_256 -n256 --ruby 
>>> --cpu-type=TimingSimpleCPU --caches --l2cache --l2_size=16MB --num-l2=256 
>>> --l1d_size=64kB --l1i_size=64kB --l1d_assoc=2 --l1i_assoc=2 
>>> --network=garnet2.0 --mem-size=4GB  --num-dirs=256 --topology=Mesh_XY 
>>> --mesh-rows=16 --options=256 --cpu-clock=1GHz
>>> 
>>> Plus, when I used the x86_AtomicSimpleCPU_template.xml, it provides me the 
>>> below errors-
>>> 
>>> Traceback (most recent call last):
>>>   File "./GEM5ToMcPAT.py", line 181, in 
>>> main()
>>>   File "./GEM5ToMcPAT.py", line 53, in main
>>> dumpMcpatOut(opts.out)
>>>   File "./GEM5ToMcPAT.py", line 80, in dumpMcpatOut
>>> exprs[i] = str(eval(exprs[i]))
>>>   File "", line 1
>>> {u'name': None
>>> ^
>>> SyntaxError: unexpected EOF while parsing
>>> 
>>> 
>>> 
>>> Thanks a lot in advance.
>>> 
>>> Yours faithfully,
>>> 
>>> F.A. Faisal
>>> 
>>> School of Information Science, JAIST
>>> Member of Inoguchi Lab
>>> Mob: 080-3048-7558
>>> 
 On Fri, Jan 26, 2018 at 8:37 PM, Andreas Brokalakis  
 wrote:
 Dear all,
 
 here: https://github.com/H2020-COSSIM/cMcPAT  you can find a slightly 
 modified version of McPAT along with conversion 

Re: [gem5-users] Script to convert Gem5 output to McPAT

[F. A. Faisal]

Dear Mr. Andreas,

Thanks a lot for your scripts on mcpat.
However, I had tried to convert my stats file and config file to a new xml
file using the GEM5ToMcPAT.py script.
Where, I found that the initial mcpat xml file remained unchanged as the
output file even I used the converting script file of GEM5ToMcPAT.py and
even no error had been shown other than the lots of warnings.
Please let me know, if I am missing some thing.

./GEM5ToMcPAT.py ../../moriya/gem5/m5out/stats.txt
../../moriya/gem5/m5out/config.json
../mcpat/ProcessorDescriptionFiles/Alpha21364.xml -o Mesh256MG4GB.xml
./GEM5ToMcPAT.py ../../moriya/gem5/m5out/stats.txt
../../moriya/gem5/m5out/config.json
../mcpat/ProcessorDescriptionFiles/Xeon.xml -o Mesh256MG4GB.xml

build/X86_MESI_Two_Level/gem5.opt configs/example/se.py -c
../nas/NPB3.3.1/NPB3.3-OMP/bin/mg.S.x_256 -n256 --ruby
--cpu-type=TimingSimpleCPU --caches --l2cache --l2_size=16MB --num-l2=256
--l1d_size=64kB --l1i_size=64kB --l1d_assoc=2 --l1i_assoc=2
--network=garnet2.0 --mem-size=4GB  --num-dirs=256 --topology=Mesh_XY
--mesh-rows=16 --options=256 --cpu-clock=1GHz

Plus, when I used the x86_AtomicSimpleCPU_template.xml, it provides me the
below errors-

Traceback (most recent call last):
  File "./GEM5ToMcPAT.py", line 181, in 
main()
  File "./GEM5ToMcPAT.py", line 53, in main
dumpMcpatOut(opts.out)
  File "./GEM5ToMcPAT.py", line 80, in dumpMcpatOut
exprs[i] = str(eval(exprs[i]))
  File "", line 1
{u'name': None
^
SyntaxError: unexpected EOF while parsing

NB- I also can share my stats file and json file, if you want in your
private e-mail.

Thanks a lot in advance.

Yours faithfully,

F.A. Faisal

School of Information Science, JAIST
Member of Inoguchi Lab
Mob: 080-3048-7558

On Fri, Feb 16, 2018 at 2:59 PM, F. A. Faisal  wrote:

> Dear Mr. Andreas,
>
> Thanks a lot for your scripts on mcpat.
> However, I had tried to convert my stats file and config file to a new xml
> file using the GEM5ToMcPAT.py script.
> Where, I found that the initial mcpat xml file remained unchanged as the
> output file even I used the converting script file of GEM5ToMcPAT.py and
> even no error had been shown other than the lots of warnings.
> Please let me know, if I am missing some thing.
>
> ./GEM5ToMcPAT.py ../../moriya/gem5/m5out/stats.txt
> ../../moriya/gem5/m5out/config.json 
> ../mcpat/ProcessorDescriptionFiles/Alpha21364.xml
> -o Mesh256MG4GB.xml
> ./GEM5ToMcPAT.py ../../moriya/gem5/m5out/stats.txt
> ../../moriya/gem5/m5out/config.json 
> ../mcpat/ProcessorDescriptionFiles/Xeon.xml
> -o Mesh256MG4GB.xml
>
> build/X86_MESI_Two_Level/gem5.opt configs/example/se.py -c
> ../nas/NPB3.3.1/NPB3.3-OMP/bin/mg.S.x_256 -n256 --ruby
> --cpu-type=TimingSimpleCPU --caches --l2cache --l2_size=16MB --num-l2=256
> --l1d_size=64kB --l1i_size=64kB --l1d_assoc=2 --l1i_assoc=2
> --network=garnet2.0 --mem-size=4GB  --num-dirs=256 --topology=Mesh_XY
> --mesh-rows=16 --options=256 --cpu-clock=1GHz
>
> Plus, when I used the x86_AtomicSimpleCPU_template.xml, it provides me
> the below errors-
>
> Traceback (most recent call last):
>   File "./GEM5ToMcPAT.py", line 181, in 
> main()
>   File "./GEM5ToMcPAT.py", line 53, in main
> dumpMcpatOut(opts.out)
>   File "./GEM5ToMcPAT.py", line 80, in dumpMcpatOut
> exprs[i] = str(eval(exprs[i]))
>   File "", line 1
> {u'name': None
> ^
> SyntaxError: unexpected EOF while parsing
>
>
>
> Thanks a lot in advance.
>
> Yours faithfully,
>
> F.A. Faisal
> 
> School of Information Science, JAIST
> Member of Inoguchi Lab
> Mob: 080-3048-7558
>
> On Fri, Jan 26, 2018 at 8:37 PM, Andreas Brokalakis 
> wrote:
>
>> Dear all,
>>
>> here: https://github.com/H2020-COSSIM/cMcPAT  you can find a slightly
>> modified version of McPAT along with conversion scripts to connect GEM5 and
>> McPAT.
>>
>> The differences between McPAT v1.3 and cMcPAT are documented. The
>> conversion scripts are based on the source code that you mention, however
>> they are modified and updated so that no errors arise from using the newer
>> versions of GEM5 or when using simpleCPU models that there are issues
>> between gem5 outputs and what McPAT requires. Additionally you can find xml
>> templates for McPAT that can be used when x86 and ARM simpleCPU models are
>> used in gem5.
>>
>> Hope it proves useful.
>>
>> Best,
>> Andreas
>>
>> On Wed, Jan 24, 2018 at 4:25 PM, Gus Smith 
>> wrote:
>>
>>> You're going to have to debug the Python script and make changes
>>> yourself. I used that same tool and had some issues. In this specific
>>> instance, it looks like the statLine object (which I assume is a regex
>>> object) is not finding any matches in the line object (which I assume is a
>>> string). Thus, None is returned. When group() is called on None, that error
>>> is thrown.
>>>
>>> I didn't hit this specific issue with the script, but I did "fix" (not
>>> 

Re: [gem5-users] Script to convert Gem5 output to McPAT

[Andreas Brokalakis]

Dear all,

here: https://github.com/H2020-COSSIM/cMcPAT  you can find a slightly
modified version of McPAT along with conversion scripts to connect GEM5 and
McPAT.

The differences between McPAT v1.3 and cMcPAT are documented. The
conversion scripts are based on the source code that you mention, however
they are modified and updated so that no errors arise from using the newer
versions of GEM5 or when using simpleCPU models that there are issues
between gem5 outputs and what McPAT requires. Additionally you can find xml
templates for McPAT that can be used when x86 and ARM simpleCPU models are
used in gem5.

Hope it proves useful.

Best,
Andreas

On Wed, Jan 24, 2018 at 4:25 PM, Gus Smith  wrote:

> You're going to have to debug the Python script and make changes yourself.
> I used that same tool and had some issues. In this specific instance, it
> looks like the statLine object (which I assume is a regex object) is not
> finding any matches in the line object (which I assume is a string). Thus,
> None is returned. When group() is called on None, that error is thrown.
>
> I didn't hit this specific issue with the script, but I did "fix" (not
> really a great fix) another issue. Feel free to apply the following patch.
>
> diff --git a/GEM5ToMcPAT.py b/GEM5ToMcPAT.py
> index 8db9674..8be8a32 100755
> --- a/GEM5ToMcPAT.py
> +++ b/GEM5ToMcPAT.py
> @@ -73,7 +73,11 @@ def dumpMcpatOut(outFile):
>  exprs[i] = str(eval(exprs[i]))
>  param.attrib['value'] = ','.join(exprs)
>  else:
> -param.attrib['value'] = str(eval(str(value)))
> +try:
> +param.attrib['value'] = str(eval(str(value)))
> +except:
> +print("error: " + str(value))
> +param.attrib['value'] = "0"
>
>  #replace stats with values from the GEM5 stats file
>  statRe = re.compile(r'stats\.([a-zA-Z0-9_:\.]+)')
>
> ​Good luck!​
>
> Gus Smith
>
> ___
> gem5-users mailing list
> gem5-users@gem5.org
> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
>
___
gem5-users mailing list
gem5-users@gem5.org
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users

Re: [gem5-users] Script to convert Gem5 output to McPAT

[Gus Smith]

You're going to have to debug the Python script and make changes yourself.
I used that same tool and had some issues. In this specific instance, it
looks like the statLine object (which I assume is a regex object) is not
finding any matches in the line object (which I assume is a string). Thus,
None is returned. When group() is called on None, that error is thrown.

I didn't hit this specific issue with the script, but I did "fix" (not
really a great fix) another issue. Feel free to apply the following patch.

diff --git a/GEM5ToMcPAT.py b/GEM5ToMcPAT.py
index 8db9674..8be8a32 100755
--- a/GEM5ToMcPAT.py
+++ b/GEM5ToMcPAT.py
@@ -73,7 +73,11 @@ def dumpMcpatOut(outFile):
 exprs[i] = str(eval(exprs[i]))
 param.attrib['value'] = ','.join(exprs)
 else:
-param.attrib['value'] = str(eval(str(value)))
+try:
+param.attrib['value'] = str(eval(str(value)))
+except:
+print("error: " + str(value))
+param.attrib['value'] = "0"

 #replace stats with values from the GEM5 stats file
 statRe = re.compile(r'stats\.([a-zA-Z0-9_:\.]+)')

​Good luck!​

Gus Smith
___
gem5-users mailing list
gem5-users@gem5.org
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users