On 13/02/15 08:39, Mark Wynter wrote:
I’ve encountered a bottleneck somewhere with v.net http://v.net when
scaling out with GNU Parallel… not sure if its an underlying issue with
v.net http://v.net or the way I’m calling the batch jobs?
I’ve got 32 CPUs and commensurate RAM. What I’m
On 13/02/15 13:40, Mark Wynter wrote:
Hi Moritz
With the second approach (the code I shared in my post), I have 3500 discrete
jobs, and I set the number of batches equal to the number of CPUs. Each batch
job is despatched to a cpu, where it then pulls from a queue of job id’s that
are
Hi Moritz, Stefan (and Marcus if you’re around?)
Every day is a new day.
Not IOPs, Not memory, Not CPU… Hmm, how about reboot...
As you have suspected, I get no benefit from additional CPUs.
Are you sure the problem is CPU-bound ?
I started by testing v.net (the maintenance module) in
Hi Moritz
With the second approach (the code I shared in my post), I have 3500 discrete
jobs, and I set the number of batches equal to the number of CPUs. Each batch
job is despatched to a cpu, where it then pulls from a queue of job id’s that
are processed in serial within each batch job.
Getting closer to the bottom of this.
On AWS, problems occur once I scale beyond 16 CPUs.
c3.4xlarge (16 CPUs, 30GB memory), v.net scales linearly
c3.8xlarge (32 CPUs, 60GB memory), v.net doesn't scale AT ALL - the same as
having single CPU.
I don’t believe its a pg issue, as I run loads of
I’ve encountered a bottleneck somewhere with v.net when scaling out with GNU
Parallel… not sure if its an underlying issue with v.net or the way I’m calling
the batch jobs?
I’ve got 32 CPUs and commensurate RAM. What I’m observing is v.net CPU
utilisation dropping off in accordance with
Sent: 13. februar 2015 08:40
To: grass-user@lists.osgeo.org
Subject: [GRASS-user] v.net parallelisation issues
I've encountered a bottleneck somewhere with v.nethttp://v.net when scaling
out with GNU Parallel... not sure if its an underlying issue with
v.nethttp://v.net or the way I'm calling
