Re: DUCC doesn't use all available machines
On Sun, Nov 30, 2014 at 11:48 AM, Simon Hafner wrote: > 2014-11-30 7:25 GMT-06:00 Eddie Epstein : > > On Sat, Nov 29, 2014 at 4:46 PM, Simon Hafner > wrote: > > > >> I've thrown some numbers at it (doubling each) and it's running at > >> comfortable 125 procs. However, at about 6.1k of 6.5k items, the procs > >> drop down to 30. > >> > > > > 125 processes at 8 threads each = 1000 active pipelines. How CPU cores > > are these 1000 pipelines running on? > Only 60. > For CPU bound pipelines, throughput will tend to degrade with more threads than cores.
Re: DUCC doesn't use all available machines
2014-11-30 7:25 GMT-06:00 Eddie Epstein : > On Sat, Nov 29, 2014 at 4:46 PM, Simon Hafner wrote: > >> I've thrown some numbers at it (doubling each) and it's running at >> comfortable 125 procs. However, at about 6.1k of 6.5k items, the procs >> drop down to 30. >> > > 125 processes at 8 threads each = 1000 active pipelines. How CPU cores > are these 1000 pipelines running on? Only 60.
Re: DUCC doesn't use all available machines
On Sat, Nov 29, 2014 at 4:46 PM, Simon Hafner wrote: > I've thrown some numbers at it (doubling each) and it's running at > comfortable 125 procs. However, at about 6.1k of 6.5k items, the procs > drop down to 30. > 125 processes at 8 threads each = 1000 active pipelines. How CPU cores are these 1000 pipelines running on?
