Hi, I need to run about 4,000 jobs that each take around 20 seconds to complete
using: cat /tmp/parList | parallel -j 28 --eta; On my 32 core machine works OK, BUT there is a "lull" in processing every few seconds as new jobs are started, once the current crop have completed. I assume this is due to an overhead in starting jobs that is only noticable because my jobs are so short. Is there any way I could make this more efficient, so my cores are fully utilised and getting through the whole process is faster? Moon