Re: [julia-users] Sharing success: running Julia on PBS cluster across compute nodes
The goal certainly is to maintain it. I will request nlhepler to see if he can transfer the repo to JuliaLang, which will help with maintenance. -viral On Thursday, August 28, 2014 4:05:27 PM UTC+5:30, Florian Oswald wrote: I'm mentioning this because we there's a dangling issue on the topic since 24 of May: https://github.com/nlhepler/ClusterManagers.jl/issues/13 On 28 August 2014 10:53, Florian Oswald florian.osw...@gmail.com wrote: no problem about adding this to clustermanagers.jl. just one question: is that repo still maintained? it seemed there was very little activity there recently. i can submit a PR there if that's the preferred solution. On 28 August 2014 00:38, Stefan Karpinski ste...@karpinski.org wrote: That's cool. It would be great if we can extract the parts of this that are not specific to the particular system you're running on and generic to PBS and added it to the ClusterManagers package. Thanks also for reporting your success story – it's always nice to hear them, regardless of magnitude :-) On Wed, Aug 27, 2014 at 6:38 PM, Florian Oswald florian.osw...@gmail.com wrote: Dear All, after bugging this list long enough with questions about how to get Julia running in parallel on a Torque/PBS managed cluster I thought I'd share my experience with the list. I realise that by julia standards this is a rather modest achievement, but I'd been happy to come across something like this post a while ago. :-) So, there's nothing special about the cluster being PBS managed (rather than SGE or whatever), I just found that each system is as idiosyncratic as the sysadmin person who set it up (starting from the format of nodenames to scheduler options to how environment variables get forwarded into a node, etc etc), so it always takes a fair amount of hacking to get something running. It's pretty low quality hacking I would say, but it's painful on a cluster. You basically need to adapt the functions in iridis_launcher.jl in the below repo to your system. Very few of the issues actually had anything to do with Julia itself, so I tried to explain as much about the environment as possible. Again, just sharing this in the hope someone out there is trying to achieve something similar may find this useful: https://github.com/floswald/parallelTest/tree/master/julia/iridis
Re: [julia-users] Sharing success: running Julia on PBS cluster across compute nodes
no problem about adding this to clustermanagers.jl. just one question: is that repo still maintained? it seemed there was very little activity there recently. i can submit a PR there if that's the preferred solution. On 28 August 2014 00:38, Stefan Karpinski ste...@karpinski.org wrote: That's cool. It would be great if we can extract the parts of this that are not specific to the particular system you're running on and generic to PBS and added it to the ClusterManagers package. Thanks also for reporting your success story – it's always nice to hear them, regardless of magnitude :-) On Wed, Aug 27, 2014 at 6:38 PM, Florian Oswald florian.osw...@gmail.com wrote: Dear All, after bugging this list long enough with questions about how to get Julia running in parallel on a Torque/PBS managed cluster I thought I'd share my experience with the list. I realise that by julia standards this is a rather modest achievement, but I'd been happy to come across something like this post a while ago. :-) So, there's nothing special about the cluster being PBS managed (rather than SGE or whatever), I just found that each system is as idiosyncratic as the sysadmin person who set it up (starting from the format of nodenames to scheduler options to how environment variables get forwarded into a node, etc etc), so it always takes a fair amount of hacking to get something running. It's pretty low quality hacking I would say, but it's painful on a cluster. You basically need to adapt the functions in iridis_launcher.jl in the below repo to your system. Very few of the issues actually had anything to do with Julia itself, so I tried to explain as much about the environment as possible. Again, just sharing this in the hope someone out there is trying to achieve something similar may find this useful: https://github.com/floswald/parallelTest/tree/master/julia/iridis
Re: [julia-users] Sharing success: running Julia on PBS cluster across compute nodes
I'm mentioning this because we there's a dangling issue on the topic since 24 of May: https://github.com/nlhepler/ClusterManagers.jl/issues/13 On 28 August 2014 10:53, Florian Oswald florian.osw...@gmail.com wrote: no problem about adding this to clustermanagers.jl. just one question: is that repo still maintained? it seemed there was very little activity there recently. i can submit a PR there if that's the preferred solution. On 28 August 2014 00:38, Stefan Karpinski ste...@karpinski.org wrote: That's cool. It would be great if we can extract the parts of this that are not specific to the particular system you're running on and generic to PBS and added it to the ClusterManagers package. Thanks also for reporting your success story – it's always nice to hear them, regardless of magnitude :-) On Wed, Aug 27, 2014 at 6:38 PM, Florian Oswald florian.osw...@gmail.com wrote: Dear All, after bugging this list long enough with questions about how to get Julia running in parallel on a Torque/PBS managed cluster I thought I'd share my experience with the list. I realise that by julia standards this is a rather modest achievement, but I'd been happy to come across something like this post a while ago. :-) So, there's nothing special about the cluster being PBS managed (rather than SGE or whatever), I just found that each system is as idiosyncratic as the sysadmin person who set it up (starting from the format of nodenames to scheduler options to how environment variables get forwarded into a node, etc etc), so it always takes a fair amount of hacking to get something running. It's pretty low quality hacking I would say, but it's painful on a cluster. You basically need to adapt the functions in iridis_launcher.jl in the below repo to your system. Very few of the issues actually had anything to do with Julia itself, so I tried to explain as much about the environment as possible. Again, just sharing this in the hope someone out there is trying to achieve something similar may find this useful: https://github.com/floswald/parallelTest/tree/master/julia/iridis
[julia-users] Sharing success: running Julia on PBS cluster across compute nodes
Dear All, after bugging this list long enough with questions about how to get Julia running in parallel on a Torque/PBS managed cluster I thought I'd share my experience with the list. I realise that by julia standards this is a rather modest achievement, but I'd been happy to come across something like this post a while ago. :-) So, there's nothing special about the cluster being PBS managed (rather than SGE or whatever), I just found that each system is as idiosyncratic as the sysadmin person who set it up (starting from the format of nodenames to scheduler options to how environment variables get forwarded into a node, etc etc), so it always takes a fair amount of hacking to get something running. It's pretty low quality hacking I would say, but it's painful on a cluster. You basically need to adapt the functions in iridis_launcher.jl in the below repo to your system. Very few of the issues actually had anything to do with Julia itself, so I tried to explain as much about the environment as possible. Again, just sharing this in the hope someone out there is trying to achieve something similar may find this useful: https://github.com/floswald/parallelTest/tree/master/julia/iridis
Re: [julia-users] Sharing success: running Julia on PBS cluster across compute nodes
That's cool. It would be great if we can extract the parts of this that are not specific to the particular system you're running on and generic to PBS and added it to the ClusterManagers package. Thanks also for reporting your success story – it's always nice to hear them, regardless of magnitude :-) On Wed, Aug 27, 2014 at 6:38 PM, Florian Oswald florian.osw...@gmail.com wrote: Dear All, after bugging this list long enough with questions about how to get Julia running in parallel on a Torque/PBS managed cluster I thought I'd share my experience with the list. I realise that by julia standards this is a rather modest achievement, but I'd been happy to come across something like this post a while ago. :-) So, there's nothing special about the cluster being PBS managed (rather than SGE or whatever), I just found that each system is as idiosyncratic as the sysadmin person who set it up (starting from the format of nodenames to scheduler options to how environment variables get forwarded into a node, etc etc), so it always takes a fair amount of hacking to get something running. It's pretty low quality hacking I would say, but it's painful on a cluster. You basically need to adapt the functions in iridis_launcher.jl in the below repo to your system. Very few of the issues actually had anything to do with Julia itself, so I tried to explain as much about the environment as possible. Again, just sharing this in the hope someone out there is trying to achieve something similar may find this useful: https://github.com/floswald/parallelTest/tree/master/julia/iridis
