Bill thanks for the advice. I will send an update when I get this working. The RenderPal software seems quite well behaved. There is a configuration file which gives the name of the render executable. it is easy to substitute this for a wrapper script. The wrapper script says that it is called with some easily parseable arguments, which are just the argument list for Blender (Maya, etc.)
________________________________________ From: Bill Broadley [[email protected]] Sent: 28 January 2017 02:37 To: slurm-dev Subject: [slurm-dev] Re: Slurm for render farm On 01/16/2017 10:02 AM, John Hearns wrote: > The concept at the moment is to run the Renderpal server, which is a Windows > application and it can detect the Linux render clients via a 'heartbeat' > mechanism. > I would spawn the Linux clients as needed via slurm. Sounds reasonable. > Thinking out loud, I could use slurm to run render clients on all compute > nodes > in the cluster, then use job preemption to kill the jobs when other compute > jobs > need the nodes. I guess that very much risks 'live' Renderpal jobs being > killed > off. This works well, but slurm doesn't understand swap. So it worked for me in testing, until I used it in production and couldn't figure out why jobs wouldn't suspend. For that reason I had to use CR_CPU instead of CR_CPU_MEM to tell slurm to only manage CPUs and not CPUs + ram. Seems kinda weird to allow suspend, but not swap. Ram is pretty expensive to waste on suspended processes. So I can't mix high memory/low cpu jobs and low memory/high CPU jobs and efficiently utilize nodes. > The script will have to block until the job completes I think - else the > RenderPal server will report it has finished. > Is it possible to block and wait till an sbatch has finished? > Or shoudl I be thinking on using srun here? Jobs could depend on other jobs, so that could work. Your jobs could do something like touch ~/job/finished/$SLURM_JOBID. But I do think the easiest would be srun. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. Employees of XMA Ltd are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to company policy and outside the scope of the employment of the individual concerned. The company will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. XMA Limited is registered in England and Wales (registered no. 2051703). Registered Office: Wilford Industrial Estate, Ruddington Lane, Wilford, Nottingham, NG11 7EP
