[gem5-users] Re: Setting up cluster for gem5

2020-11-06 Thread Shehab Elsayed via gem5-users
Thank you so much for your reply, Daniel! It is really helpful.


On Fri, Nov 6, 2020 at 10:21 AM Daniel Gerzhoy 
wrote:

> Hey Shehab,
>
> I've been working with gem5 on my group's research cluster for a while
> now.
> 1) Gem5 isn't very memory hungry in my experience, sometimes long
> simulations (I'm talking 3 weeks+) will  start bloating to GB of RAM but
> its usually not paging so it doesn't slow things down (depends on the
> program you are running)
> I exclusively use Syscall Emulation mode, so that may not apply in Full
> System.
>
> *It is however single-threaded. So if your entire group is running many
> experiments at the same time make sure you have a ton of cores.*
>
> 2) As for job management, I created my own system for
> configuring/running/parsing etc.that I've built with python.
>
> gem5 as of recently has been shipped with dockerfiles. I use the gcn3
> dockerfile for instance. I'd recommend using them.
> Again I use a custom solution here, but I'm pretty sure container job
> management is a solved problem. I think one of them is "kubernets" (see
> https://kubernetes.io/)
> I don't have experience with anything like that, but I'm sure that would
> be useful.
>
> Also, if you plan on editing gem5 and your sourcecode is going to be
> located on the cluster, I'd recommend using code-server (
> https://github.com/cdr/code-server)
> It broadcasts an instance of vscode to a web page that you can access from
> anywhere. I used to use gvim and bash scripts and it was hell. Code server
> was a life-saver.
>
> If you (or anyone else) already have a solution for editing code on the
> cluster I would be interested in what it is.
>
> Good luck!
>
> Dan Gerzhoy
> PhD Candidate, Computer Engineering
> University of Maryland College Park
>
> On Fri, Nov 6, 2020 at 8:38 AM Shehab Elsayed via gem5-users <
> gem5-users@gem5.org> wrote:
>
>> Hello All,
>>
>> My group is in the process of upgrading our cluster and since many of us
>> are using gem5 I was wondering if anyone has experience or recommendation
>> they would like to share about the process for a smooth gem5 operation.
>> Mainly I am concerned about 2 issues:
>>
>> 1) Required hard disk and memory on the nodes for a smooth gem5
>> operation.
>> 2) OS and job management systems or any software related recommendations.
>>
>> Thank you very much in advance.
>>
>> Best Regards,
>> Shehab
>> ___
>> gem5-users mailing list -- gem5-users@gem5.org
>> To unsubscribe send an email to gem5-users-le...@gem5.org
>> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
>
>
___
gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

[gem5-users] Re: Setting up cluster for gem5

2020-11-06 Thread Daniel Gerzhoy via gem5-users
Hey Shehab,

I've been working with gem5 on my group's research cluster for a while now.
1) Gem5 isn't very memory hungry in my experience, sometimes long
simulations (I'm talking 3 weeks+) will  start bloating to GB of RAM but
its usually not paging so it doesn't slow things down (depends on the
program you are running)
I exclusively use Syscall Emulation mode, so that may not apply in Full
System.

*It is however single-threaded. So if your entire group is running many
experiments at the same time make sure you have a ton of cores.*

2) As for job management, I created my own system for
configuring/running/parsing etc.that I've built with python.

gem5 as of recently has been shipped with dockerfiles. I use the gcn3
dockerfile for instance. I'd recommend using them.
Again I use a custom solution here, but I'm pretty sure container job
management is a solved problem. I think one of them is "kubernets" (see
https://kubernetes.io/)
I don't have experience with anything like that, but I'm sure that would be
useful.

Also, if you plan on editing gem5 and your sourcecode is going to be
located on the cluster, I'd recommend using code-server (
https://github.com/cdr/code-server)
It broadcasts an instance of vscode to a web page that you can access from
anywhere. I used to use gvim and bash scripts and it was hell. Code server
was a life-saver.

If you (or anyone else) already have a solution for editing code on the
cluster I would be interested in what it is.

Good luck!

Dan Gerzhoy
PhD Candidate, Computer Engineering
University of Maryland College Park

On Fri, Nov 6, 2020 at 8:38 AM Shehab Elsayed via gem5-users <
gem5-users@gem5.org> wrote:

> Hello All,
>
> My group is in the process of upgrading our cluster and since many of us
> are using gem5 I was wondering if anyone has experience or recommendation
> they would like to share about the process for a smooth gem5 operation.
> Mainly I am concerned about 2 issues:
>
> 1) Required hard disk and memory on the nodes for a smooth gem5 operation.
> 2) OS and job management systems or any software related recommendations.
>
> Thank you very much in advance.
>
> Best Regards,
> Shehab
> ___
> gem5-users mailing list -- gem5-users@gem5.org
> To unsubscribe send an email to gem5-users-le...@gem5.org
> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
___
gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s