On 7 February 2015 at 14:23, Ole Tange <ta...@gnu.org> wrote: > On Sat, Feb 7, 2015 at 10:11 AM, xmoon 2000 <xmoon2...@googlemail.com> wrote: >> On 6 February 2015 at 20:42, Ole Tange <o...@tange.dk> wrote: >>> On Fri, Feb 6, 2015 at 3:56 PM, xmoon 2000 <xmoon2...@googlemail.com> wrote: >>> >>> My experience with Cygwin is limited, but I seem to remember that >>> spawning is expensive (in the order of 0.1 second), which is one of >>> the reasons for recommending MSYS. So if your jobs all finish at the >>> same time, GNU Parallel will need 3.2 seconds to start 32 new jobs. >>> Maybe you will have a better experience if you delay starting jobs by >>> 0.1 seconds: >>> >>> cat /tmp/parList | parallel -j 28 --delay 0.1 --eta; >>> >>> What is your reason for not using all 32 cores? >> >> 1. Reason for not using 32 cores is then I can't do anything else. I >> need to do emails, check web, simple stuff whilst waiting for jobs to >> finish. > > I tested this on a 4 core Microsoft Windows 8 machine and it is not > true there: The machine will be slower, but not useless. On a 32 core > machine I would expect this to be even less of a problem. > > On GNU/Linux you would use --nice which would mean that all other > processes would get more CPU time than the jobs. In practice the jobs > will only use the spare CPU time. I have not tested if --nice gives > the same behaviour on Microsoft Windows. > > If you simply assumed your machine would be slower, then I will > encourage you to try it out. You might be pleasantly surprised. > > If it is possible for you to use MSYS instead of Cygwin it would be > interesting to see if that makes the problem go away. > > > /Ole
Ole, On my machine, I get 2 issues: 1. If allowing 50 core to be used - I get very slow response times for anything else I do. Slw enough to be not workable. 2. I aslo get "Resource busy" & "fork failed" message. I think this is because I am starting too many forked processes. Each jobs I start might have 3 or 4 forks. I suspect cygwin/windows is running out of handles or something
