Use the cores you have : 32 Do you still get those issues?
Cheers, Felipe On Wed, Feb 11, 2015 at 12:55 PM, xmoon 2000 <xmoon2...@googlemail.com> wrote: > On 7 February 2015 at 14:23, Ole Tange <ta...@gnu.org> wrote: > > On Sat, Feb 7, 2015 at 10:11 AM, xmoon 2000 <xmoon2...@googlemail.com> > wrote: > >> On 6 February 2015 at 20:42, Ole Tange <o...@tange.dk> wrote: > >>> On Fri, Feb 6, 2015 at 3:56 PM, xmoon 2000 <xmoon2...@googlemail.com> > wrote: > >>> > >>> My experience with Cygwin is limited, but I seem to remember that > >>> spawning is expensive (in the order of 0.1 second), which is one of > >>> the reasons for recommending MSYS. So if your jobs all finish at the > >>> same time, GNU Parallel will need 3.2 seconds to start 32 new jobs. > >>> Maybe you will have a better experience if you delay starting jobs by > >>> 0.1 seconds: > >>> > >>> cat /tmp/parList | parallel -j 28 --delay 0.1 --eta; > >>> > >>> What is your reason for not using all 32 cores? > >> > >> 1. Reason for not using 32 cores is then I can't do anything else. I > >> need to do emails, check web, simple stuff whilst waiting for jobs to > >> finish. > > > > I tested this on a 4 core Microsoft Windows 8 machine and it is not > > true there: The machine will be slower, but not useless. On a 32 core > > machine I would expect this to be even less of a problem. > > > > On GNU/Linux you would use --nice which would mean that all other > > processes would get more CPU time than the jobs. In practice the jobs > > will only use the spare CPU time. I have not tested if --nice gives > > the same behaviour on Microsoft Windows. > > > > If you simply assumed your machine would be slower, then I will > > encourage you to try it out. You might be pleasantly surprised. > > > > If it is possible for you to use MSYS instead of Cygwin it would be > > interesting to see if that makes the problem go away. > > > > > > /Ole > > Ole, > > On my machine, I get 2 issues: > > 1. If allowing 50 core to be used - I get very slow response times for > anything else I do. Slw enough to be not workable. > > 2. I aslo get "Resource busy" & "fork failed" message. I think this is > because I am starting too many forked processes. Each jobs I start > might have 3 or 4 forks. I suspect cygwin/windows is running out of > handles or something > >
