Re: Problem with parallel : it stops with a warning after several hours of operations

2017-07-11 Thread Ole Tange 
On Tue, Jul 11, 2017 at 9:35 AM, Shlomi Fish  wrote:
> On Tue, 11 Jul 2017 08:50:49 +0200 (CEST)
> Hubert Kowalski  wrote:
>
>> > parallel: Warning: No more file handles.
>> > parallel: Warning: Raising ulimit -n or /etc/security/limits.conf may
>> > help.
>>
>> Parallel runs out of file handles. try railing limit, for example
>> https://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
:
> But why does it happen?

That is unclear at the moment. It is not a known GNU Parallel bug.

> I only run 4 jobs at the same time. Aren't file handles getting closed and 
> recycled?

Yes.

The only theory I currently have is that your system is hitting the
global limit for files open - not the local per-session-limit.

So if there is something else running on the system, that uses a lot
of file handles, then this can explain what you see.

> I don't think it should happen.

See if you can file a bug as per REPORTING BUGS in the man page.

Especially important is the part on minimal, complete, and verifiable
example. If others cannot reproduce your findings, then it is
extremely hard to debug.

See if you can make the error show up on a different system.


/Ole

Re: Problem with parallel : it stops with a warning after several hours of operations

2017-07-11 Thread Hubert Kowalski 
> parallel: Warning: No more file handles.
> parallel: Warning: Raising ulimit -n or /etc/security/limits.conf may help.

Parallel runs out of file handles. try railing limit, for example 
https://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/

hope it helps.

> Dnia 10 lipiec 2017 o 17:43 Shlomi Fish  napisał(a):
>
>
> Hi all!
>
> With parallel-20170522-1.mga6 on Mageia v6 x86-64, I am getting this:
>
> 1,499,693,878.343569040 == 5,645,356,909 ==
> 1,499,693,878.343580961 I could not solve this game.
> 1,499,693,878.343592882 Total number of states checked is 310,498.
> 1,499,693,878.343604088 [[== End 5,645,356,909 ==]]
> 1,499,693,890.266119957 == 5,645,357,053 ==
> 1,499,693,890.266165018 Iterations count exceeded.
> 1,499,693,890.266180992 Total number of states checked is 2,400,000.
> 1,499,693,890.266192913 [[== End 5,645,357,053 ==]]
> 1,499,693,890.266206026 == 5,645,359,103 ==
> 1,499,693,890.266216993 I could not solve this game.
> 1,499,693,890.266228914 Total number of states checked is 1,117,997.
> 1,499,693,890.266239882 [[== End 5,645,359,103 ==]]
> 1,499,693,891.484687090 == 5,645,384,850 ==
> 1,499,693,891.484735966 Iterations count exceeded.
> 1,499,693,891.484750986 Total number of states checked is 2,400,000.
> 1,499,693,891.484764099 [[== End 5,645,384,850 ==]]
> parallel: Warning: No more file handles.
> parallel: Warning: Raising ulimit -n or /etc/security/limits.conf may help.
>
> 
>
> and parallel stops after that.
>
> It is done after running it using
> https://github.com/shlomif/freecell-3fc-determine-solvability and with -j4 on 
> a
> Core i3 machine. Is it a bug ? Can it be fixed?
>
> Regards,
>
> Shlomi Fish
>
> --
> -
> Shlomi Fish http://www.shlomifish.org/
> What does “Zionism” mean? - http://shlom.in/def-zionism
>
> Summer Glau can restore the people that Chuck killed, back to life.
> — http://www.shlomifish.org/humour/bits/facts/Summer-Glau/
>
> Please reply to list if it's a mailing list post - http://shlom.in/reply .
>

Re: Problem with parallel : it stops with a warning after several hours of operations

2017-07-11 Thread Hubert Kowalski 
Maybe something is leaking file handles? one would neet to strace this

> Dnia 11 lipiec 2017 o 09:35 Shlomi Fish  napisał(a):
>
>
> Hi Hubert,
>
> thanks for your reply.
>
> On Tue, 11 Jul 2017 08:50:49 +0200 (CEST)
> Hubert Kowalski  wrote:
>
> > > parallel: Warning: No more file handles.
> > > parallel: Warning: Raising ulimit -n or /etc/security/limits.conf may
> > > help.
> >
> > Parallel runs out of file handles. try railing limit, for example
> > https://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
> >
> > hope it helps.
> >
>
> But why does it happen? I only run 4 jobs at the same time. Aren't file 
> handles
> getting closed and recycled? I don't think it should happen.
>
> Regards,
>
> Shlomi
>
> > > Dnia 10 lipiec 2017 o 17:43 Shlomi Fish  
> > > napisał(a):
> > >
> > >
> > > Hi all!
> > >
> > > With parallel-20170522-1.mga6 on Mageia v6 x86-64, I am getting this:
> > >
> > > 1,499,693,878.343569040 == 5,645,356,909 ==
> > > 1,499,693,878.343580961 I could not solve this game.
> > > 1,499,693,878.343592882 Total number of states checked is 310,498.
> > > 1,499,693,878.343604088 [[== End 5,645,356,909 ==]]
> > > 1,499,693,890.266119957 == 5,645,357,053 ==
> > > 1,499,693,890.266165018 Iterations count exceeded.
> > > 1,499,693,890.266180992 Total number of states checked is 2,400,000.
> > > 1,499,693,890.266192913 [[== End 5,645,357,053 ==]]
> > > 1,499,693,890.266206026 == 5,645,359,103 ==
> > > 1,499,693,890.266216993 I could not solve this game.
> > > 1,499,693,890.266228914 Total number of states checked is 1,117,997.
> > > 1,499,693,890.266239882 [[== End 5,645,359,103 ==]]
> > > 1,499,693,891.484687090 == 5,645,384,850 ==
> > > 1,499,693,891.484735966 Iterations count exceeded.
> > > 1,499,693,891.484750986 Total number of states checked is 2,400,000.
> > > 1,499,693,891.484764099 [[== End 5,645,384,850 ==]]
> > > parallel: Warning: No more file handles.
> > > parallel: Warning: Raising ulimit -n or /etc/security/limits.conf may 
> > > help.
> > >
> > > 
> > >
> > > and parallel stops after that.
> > >
> > > It is done after running it using
> > > https://github.com/shlomif/freecell-3fc-determine-solvability and with -j4
> > > on a Core i3 machine. Is it a bug ? Can it be fixed?
> > >
> > > Regards,
> > >
> > > Shlomi Fish
> > >
> > > --
> > > -
> > > Shlomi Fish http://www.shlomifish.org/
> > > What does “Zionism” mean? - http://shlom.in/def-zionism
> > >
> > > Summer Glau can restore the people that Chuck killed, back to life.
> > > — http://www.shlomifish.org/humour/bits/facts/Summer-Glau/
> > >
> > > Please reply to list if it's a mailing list post - http://shlom.in/reply .
> > >
>
>
>
> --
> -
> Shlomi Fish http://www.shlomifish.org/
> http://youtu.be/xZLwtc9x4yA - Anime in Real Life!! (Parody)
>
>  “We are NO LONGER the knights who say ‘BitKeeper’. We are now
> the knights who say ‘git, git, git, cogito — Linus!’.”
>
> Please reply to list if it's a mailing list post - http://shlom.in/reply .
>

Re: Problem with parallel : it stops with a warning after several hours of operations

2017-07-11 Thread Shlomi Fish 
Hi Hubert,

thanks for your reply.

On Tue, 11 Jul 2017 08:50:49 +0200 (CEST)
Hubert Kowalski  wrote:

> > parallel: Warning: No more file handles.
> > parallel: Warning: Raising ulimit -n or /etc/security/limits.conf may
> > help.  
> 
> Parallel runs out of file handles. try railing limit, for example
> https://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
> 
> hope it helps.
> 

But why does it happen? I only run 4 jobs at the same time. Aren't file handles
getting closed and recycled? I don't think it should happen.

Regards,

Shlomi

> > Dnia 10 lipiec 2017 o 17:43 Shlomi Fish  napisał(a):
> >
> >
> > Hi all!
> >
> > With parallel-20170522-1.mga6 on Mageia v6 x86-64, I am getting this:
> >
> > 1,499,693,878.343569040 == 5,645,356,909 ==
> > 1,499,693,878.343580961 I could not solve this game.
> > 1,499,693,878.343592882 Total number of states checked is 310,498.
> > 1,499,693,878.343604088 [[== End 5,645,356,909 ==]]
> > 1,499,693,890.266119957 == 5,645,357,053 ==
> > 1,499,693,890.266165018 Iterations count exceeded.
> > 1,499,693,890.266180992 Total number of states checked is 2,400,000.
> > 1,499,693,890.266192913 [[== End 5,645,357,053 ==]]
> > 1,499,693,890.266206026 == 5,645,359,103 ==
> > 1,499,693,890.266216993 I could not solve this game.
> > 1,499,693,890.266228914 Total number of states checked is 1,117,997.
> > 1,499,693,890.266239882 [[== End 5,645,359,103 ==]]
> > 1,499,693,891.484687090 == 5,645,384,850 ==
> > 1,499,693,891.484735966 Iterations count exceeded.
> > 1,499,693,891.484750986 Total number of states checked is 2,400,000.
> > 1,499,693,891.484764099 [[== End 5,645,384,850 ==]]
> > parallel: Warning: No more file handles.
> > parallel: Warning: Raising ulimit -n or /etc/security/limits.conf may help.
> >
> > 
> >
> > and parallel stops after that.
> >
> > It is done after running it using
> > https://github.com/shlomif/freecell-3fc-determine-solvability and with -j4
> > on a Core i3 machine. Is it a bug ? Can it be fixed?
> >
> > Regards,
> >
> > Shlomi Fish
> >
> > --
> > -
> > Shlomi Fish http://www.shlomifish.org/
> > What does “Zionism” mean? - http://shlom.in/def-zionism
> >
> > Summer Glau can restore the people that Chuck killed, back to life.
> > — http://www.shlomifish.org/humour/bits/facts/Summer-Glau/
> >
> > Please reply to list if it's a mailing list post - http://shlom.in/reply .
> >  



-- 
-
Shlomi Fish   http://www.shlomifish.org/
http://youtu.be/xZLwtc9x4yA - Anime in Real Life!! (Parody)

 “We are NO LONGER the knights who say ‘BitKeeper’. We are now
the knights who say ‘git, git, git, cogito — Linus!’.”

Please reply to list if it's a mailing list post - http://shlom.in/reply .