Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-17 Thread Jim Meyering
Pádraig Brady wrote: On 16/03/11 15:32, Jim Meyering wrote: Pádraig Brady wrote: With SUBTHREAD_LINES_HEURISTIC=128k and -S1M option to sort we get no threads as nlines never gets above 12787 (there looks to be around 80 bytes overhead per line?). Only when -S = 12M do we get nlines high

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-16 Thread Jim Meyering
Pádraig Brady wrote: I've not fully analyzed this yet, and I'm not saying it's wrong, but the above change seems to have a large effect on thread creation when smaller buffers are used (you hinted previously that being less aggressive with the amount of mem used by default might be

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-16 Thread Pádraig Brady
On 16/03/11 12:07, Jim Meyering wrote: Pádraig Brady wrote: I've not fully analyzed this yet, and I'm not saying it's wrong, but the above change seems to have a large effect on thread creation when smaller buffers are used (you hinted previously that being less aggressive with the amount of

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-16 Thread Pádraig Brady
On 16/03/11 15:32, Jim Meyering wrote: Pádraig Brady wrote: With SUBTHREAD_LINES_HEURISTIC=128k and -S1M option to sort we get no threads as nlines never gets above 12787 (there looks to be around 80 bytes overhead per line?). Only when -S = 12M do we get nlines high enough to create

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-14 Thread Pádraig Brady
On 12/03/11 15:34, Jim Meyering wrote: Jim Meyering wrote: Jim Meyering wrote: Jim Meyering wrote: Running make -j25 check on a nominal-12-core F14 system would cause serious difficulty leading to an OOM kill -- and this is brand new. It worked fine yesterday. I tracked it down to all of

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-14 Thread Pádraig Brady
On 14/03/11 15:31, Pádraig Brady wrote: On 12/03/11 15:34, Jim Meyering wrote: Jim Meyering wrote: Jim Meyering wrote: Jim Meyering wrote: Running make -j25 check on a nominal-12-core F14 system would cause serious difficulty leading to an OOM kill -- and this is brand new. It worked fine

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-12 Thread Jim Meyering
Jim Meyering wrote: Jim Meyering wrote: Jim Meyering wrote: Running make -j25 check on a nominal-12-core F14 system would cause serious difficulty leading to an OOM kill -- and this is brand new. It worked fine yesterday. I tracked it down to all of the make processes working on the

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-02-10 Thread Jim Meyering
Jim Meyering wrote: Jim Meyering wrote: Running make -j25 check on a nominal-12-core F14 system would cause serious difficulty leading to an OOM kill -- and this is brand new. It worked fine yesterday. I tracked it down to all of the make processes working on the built_programs.list (in