Re: [boinc_dev] boinc_dev Digest, Vol 153, Issue 3

[Jord van der Elst]

What John is looking for is called "Leave Non-GPU tasks in memory when
suspended", used to be "Leave applications in memory".
With this switched on, tasks that suspend keep their whole state in memory.
With it switched off tasks that don't checkpoint start from the beginning,
tasks that checkpoint start from the last checkpoint. But tasks that don't
checkpoint usually don't switch out of memory until they're done. They can
only switch out of memory when the user forces the task to do so (suspend),
or by quitting BOINC.


-- Jord van der Elst.

On Tue, Mar 7, 2017 at 3:09 PM, McLeod, John  wrote:

> It depends.  If you suspend a task and return all the memory, then you
> lose any forward progress since the last check point.  There is a setting
> that allows the removal from memory (or at least there used to be), but I
> don't remember what it is.  A suspended task's memory can go to swap with
> no problem as it will not be touched until the next time that the task is
> resumed.
>
> -Original Message-
> From: boinc_dev [mailto:boinc_dev-boun...@ssl.berkeley.edu] On Behalf Of
> Robert Miles
> Sent: Tuesday, March 7, 2017 8:22 AM
> To: boinc_dev@ssl.berkeley.edu
> Subject: Re: [boinc_dev] boinc_dev Digest, Vol 153, Issue 3
>
>  From what I've seen, when BOINC suspends workunits, this DOES NOT free the
> memory they are using.
>
> It looks like BOINC should not be allowed to start a workunit without
> checking
> whether enough memory is still free, and it should then be required to
> repeat
> this check before starting the next one.
> > Date: Mon, 6 Mar 2017 21:27:47 +0100
> > From: yoyo 
> > To: BOINC Developers Mailing List ,
>  BOINC
> >   Projects 
> > Subject: [boinc_dev] Boinc handling of workunits with much RAM
> >   requirements
> >
> > Hello,
> >
> > I'd like to understand what Boinc does with workunits which require a
> > large amount of RAM.
> >
> > I have workunits which require 10 GB RAM.
> > My understandig is, that only hosts which have at least 10 GB free RAM
> > are downloading them and only if 10 GB RAM are free they are started?
> >
> > It is not clear for me what happens when the workunits are running and
> > if and how often Boinc checks their RAM consumption and what Boinc does
> > if they consume too much RAM.
> >
> > A user blames, that he has a system with 8 cores and 16 GB RAM. This
> > systems has more than 10GB free RAM. So such workunits are downloaded
> > and started, 8 of them in parallel at the same time. After some seconds
> > all 8 workunits consuming 10 GB RAM each and the system is havily
> > swaping and nearly unresponsive.
> > Shouldn't Boinc find out, that too much RAM is consumed and suspend some
> > of the workunits?
> >
> > Kind regards,
> > yoyo
>
> ___
> boinc_dev mailing list
> boinc_dev@ssl.berkeley.edu
> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.
> ___
> boinc_dev mailing list
> boinc_dev@ssl.berkeley.edu
> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.
>
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] boinc_dev Digest, Vol 153, Issue 3

[McLeod, John]

It depends.  If you suspend a task and return all the memory, then you lose any 
forward progress since the last check point.  There is a setting that allows 
the removal from memory (or at least there used to be), but I don't remember 
what it is.  A suspended task's memory can go to swap with no problem as it 
will not be touched until the next time that the task is resumed.

-Original Message-
From: boinc_dev [mailto:boinc_dev-boun...@ssl.berkeley.edu] On Behalf Of Robert 
Miles
Sent: Tuesday, March 7, 2017 8:22 AM
To: boinc_dev@ssl.berkeley.edu
Subject: Re: [boinc_dev] boinc_dev Digest, Vol 153, Issue 3

 From what I've seen, when BOINC suspends workunits, this DOES NOT free the
memory they are using.

It looks like BOINC should not be allowed to start a workunit without 
checking
whether enough memory is still free, and it should then be required to 
repeat
this check before starting the next one.
> Date: Mon, 6 Mar 2017 21:27:47 +0100
> From: yoyo 
> To: BOINC Developers Mailing List ,   BOINC
>   Projects 
> Subject: [boinc_dev] Boinc handling of workunits with much RAM
>   requirements
>
> Hello,
>
> I'd like to understand what Boinc does with workunits which require a
> large amount of RAM.
>
> I have workunits which require 10 GB RAM.
> My understandig is, that only hosts which have at least 10 GB free RAM
> are downloading them and only if 10 GB RAM are free they are started?
>
> It is not clear for me what happens when the workunits are running and
> if and how often Boinc checks their RAM consumption and what Boinc does
> if they consume too much RAM.
>
> A user blames, that he has a system with 8 cores and 16 GB RAM. This
> systems has more than 10GB free RAM. So such workunits are downloaded
> and started, 8 of them in parallel at the same time. After some seconds
> all 8 workunits consuming 10 GB RAM each and the system is havily
> swaping and nearly unresponsive.
> Shouldn't Boinc find out, that too much RAM is consumed and suspend some
> of the workunits?
>
> Kind regards,
> yoyo

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] boinc_dev Digest, Vol 153, Issue 3

[Robert Miles]

From what I've seen, when BOINC suspends workunits, this DOES NOT free the
memory they are using.

It looks like BOINC should not be allowed to start a workunit without 
checking
whether enough memory is still free, and it should then be required to 
repeat

this check before starting the next one.

Date: Mon, 6 Mar 2017 21:27:47 +0100
From: yoyo 
To: BOINC Developers Mailing List ,   BOINC
Projects 
Subject: [boinc_dev] Boinc handling of workunits with much RAM
requirements

Hello,

I'd like to understand what Boinc does with workunits which require a
large amount of RAM.

I have workunits which require 10 GB RAM.
My understandig is, that only hosts which have at least 10 GB free RAM
are downloading them and only if 10 GB RAM are free they are started?

It is not clear for me what happens when the workunits are running and
if and how often Boinc checks their RAM consumption and what Boinc does
if they consume too much RAM.

A user blames, that he has a system with 8 cores and 16 GB RAM. This
systems has more than 10GB free RAM. So such workunits are downloaded
and started, 8 of them in parallel at the same time. After some seconds
all 8 workunits consuming 10 GB RAM each and the system is havily
swaping and nearly unresponsive.
Shouldn't Boinc find out, that too much RAM is consumed and suspend some
of the workunits?

Kind regards,
yoyo


___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Project application using more memory than it, reports to BOINC

[Jord van der Elst]

Windows 10 - 64bit.
But his VBox is just 5.0.32 if his signature is to be believed. I have
advised him to update to 5.1.14, so thanks for that catch.


-- Jord van der Elst.

On Tue, Mar 7, 2017 at 6:28 AM, Robert Miles 
wrote:

> Does he run 32-bit applications under 64-bit Windows Vista?  I've found
> that nearly
> all 32-bit programs running under 64-bit Windows Vista cause Windows to
> load
> enough svchost.exe modules to approximately double the amount of memory
> these
> programs use.  This happens even if these 32-bit applications are
> unrelated to BOINC.
> I have not seen this happen under Windows 7 or Windows 10.
>
> Also, which version of VirtualBox is he using?  I found that the 5.0.*
> versions do not
> work properly for the VM applications I run, but the 5.1.* versions run
> much better,
> without using a lot of extra memory.
>
> Note that he may be seeing that problem only when running VM tasks. So it
> might
> be more reasonable to turn off VM tasks on all the servers which allow it,
> drop any
> BOINC projects that don't, and then keep BOINC running for the remaining
> projects
> and possibly add a few more.
>
>> Date: Sat, 4 Mar 2017 11:24:47 +0100
>> From: Jord van der Elst 
>> Subject: [boinc_dev] Project application using more memory than it
>> reports to BOINC
>>
>> Hi David,
>>
>> A user on the BOINC forums reported that he was now getting ready to get
>> rid of BOINC, because it would use his whole memory and then even more,
>> swapping to page file and there was nothing he could do about it. (ref
>> https://boinc.berkeley.edu/dev/forum_thread.php?id=11494)
>>
>> He states to be running into this problem when running ATLAS GARL tasks,
>> that use 2GB per task, which he appeared to run 10 of at the same time,
>> using then 20GB of memory. A bit troublesome on a system using 16GB.
>>
>> So stopping that, he switched to LHC and running the LHCb simulation he
>> now
>> runs into to same problem. Here however there is a thread explaining why
>> that is (https://lhcathome.cern.ch/sixtrack/forum_thread.php?id=4072):
>> the
>> LHCb tasks use 2GB of memory but report they only use 480MB per. So BOINC
>> thinks they use 480MB, but they really do use 2GB.
>>
>> These all run in VirtualBox for as far as I understand, so perhaps that
>> there's a bug in the wrapper how the memory usage is being read. In any
>> case, shouldn't BOINC always poll the actual memory in use and stop using
>> more when all of the available RAM has been consumed, even if there are
>> free CPUs?
>>
>> Thanks,
>>
>> -- Jord van der Elst.
>>
>
>
> ___
> boinc_dev mailing list
> boinc_dev@ssl.berkeley.edu
> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.
>
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.