On Tuesday, 12 May 2015 at 14:59:38 UTC, Gerald Jansen wrote:
The output of /usr/bin/time is as follows: Lang Jobs User System Elapsed %CPU Py 2 79.24 2.16 0:48.90 166 D 2 19.41 10.14 0:17.96 164 Py 30 1255.17 58.38 2:39.54 823 * Pool(12) D 30 421.61 4565.97 6:33.73 1241
The fact that most of the time is spent in System department is quite important. I suspect there are too many system calls from line-wise reading and writing the files. How many lines are read and written there?