With the exception of vacation planning, I can see how to do the
calculation of the following: maximum duration running but not connected,
maximum duration not connected (running or not). These should follow mean
+- 3*stdev for each. However, one of the items that connect every X is
often used for is vacation planning. As in, I am going away for the
weekend, I want to leave the computers on, but they will have no network
connection.
work_buf_min_queue = maximum duration running but not connected. (we would
have to add "extra work" and hysteresis here somehow). Extra work would
either be user specified or calculated as some fixed % of
work_buf_min_queue. Override for Extra Work could be in cc_config. NOTE
that we could remove some of the time stats from the work fetch
calculation. Will have to review to figure out which ones.
computation_deadline = report deadline - (maximum duration not connected +
task switch interval + (I would like to add some project specific factors
such as max time spent running at 100% or more, time from completion to
upload and time from first attempted report to successful report)). The
items I would like to add have caused late work in the past. They collect
information about project specific problems such as tasks that have compute
time that is hidden from BOINC (superlink can run for many hours over 100%
with no checkpoints), large uploads over slow links (CPDN comes to mind
here among others), and project down time (SETI comes to mind here with the
weekly 3 day shutdown).
Factors that we have to be aware of when determining extra work and
hysteresis are:
the maximum across projects that are likely to be asked for work of the
shortest deadline within a project. We do not want to automatically create
a set of queues such that the users only project cannot meet any deadline
because the deadlines are a day and we have set extra work to be 2 days.
This means that we need to track the duration between work assignments and
deadlines at the client. This also means that "Extra work and hysteresis
can fluctuate. For example, if the next project to be asked for work has
deadlines of a day, we do not want to ask for 2 days of work from that
project to fill extra work and hysteresis. However, if the deadlines are a
month, then asking for that 2 days of hysteresis will be no problem.
I see a difference between "Extra work" and hysteresis. Extra work is
there to ensure that sufficient work is available to make it between
connections without idle resources. Hysteresis is there to aid the servers
by reducing load on the database. So these two would be added to work
fetches differently. Extra work is added to work_buf_min_queue to
determine the trigger point for a work fetch, and hysteresis is added to
the work fetch itself.
There are several possibilities for calculating hysteresis. First is to
allow it to be a project admin specified option. Some projects may need
large hysteresis in order to help their servers and others may need small
or 0 hysteresis in order to get work turned around faster. The second
method would be to use a % of work_buf_min_queue. The third would be to
use a fixed value. However, this fixed value cannot be very large as some
projects have moderately short deadlines ( a day or less).
There are several possibilities for "Extra work". A percentage of
work_buf_min_queue (can be 0 unfortunately), f*stdev(maximum duration
running but not connected). Where f is some number that makes sense (1?).
Unfortunately, this can be 0 as well. The maximum of the two. While this
can be 0 it is a bit less likely than the other. Half of my computers
would have 0 or nearly 0 for one, the other half would be similarly 0 or
nearly 0 for the other, but neither of them would be 0 for both. Another
possibility would be to have it be the based on recorded project downtime
(see a previous paragraph about wish list for computation deadline). If we
recorded the maximum downtime for each project as determined by the time
between an attempted task report and the actuality of the task report (mean
+ 3*stdev) either per project or across all projects, we could use a
calculation based on that. The two possibilities that come to mind are sum
(maximum downtime for each project)/ some calculation based on the count of
projects. There two stipulations for this calculation. If the count of
projects is 1, the result of the calculation should be 1. If the count is
> 1, the result should be > 1. However, linear seems to be a bit slow.
The second possibility is to track the maximum downtime across projects
directly. However, this has the problem that when the user detaches from a
project the calculation is no longer valid. The case when this is bad is
when a project has had difficulties with uptime (think SETI over the last 6
months with some major outages and the user has finally gotten tired of the
wait. The user gives up and finally detaches from that project - but the
downtime is still built into the extra work calculation.
In any case, the work request to a project should always be limited to
minimum duration from assignment to computation deadline * 90% - maxumum
recorded runtime for any task on that project - estimate of work already on
host for that project (suitably parameterized for the number of that
resource of course). This avoids requesting work that has little or no
chance of being returned by deadline. Note: If minimum duration from
assignment to computation deadline < maximum duration not connected, this
project is not suitable for this computer, and the user should be warned
about such. In such a case, work requests should be 1 second.
jm7
David Anderson
<[email protected]
ey.edu> To
<[email protected]>
12/09/2010 04:32 cc
PM BOINC Developers Mailing List
<[email protected]>
Subject
Re: [boinc_dev] preferences remodel
proposal
work buffer sizes would be calculated from the statistics
of network availability;
details are yet to be determined; suggestions welcome.
-- David
On 09-Dec-2010 9:09 AM, [email protected] wrote:
> I have been thinking about "Extra work". I don't see how to calculate
this
> from data. Could you give us a hint on how this would be calculated
> automatically?
>
> jm7
>
>
>
> David Anderson
> <[email protected]
> ey.edu>
To
> Sent by: BOINC Developers Mailing List
> <boinc_dev-bounce<[email protected]>
> [email protected]
cc
> u>
>
Subject
> [boinc_dev] preferences remodel
> 12/08/2010 05:35 proposal
> PM
>
>
>
>
>
>
>
>
>
> http://boinc.berkeley.edu/trac/wiki/PrefsRemodel
>
> This isn't something we're going to do anytime soon,
> but we may as well discuss it.
>
> -- David
> _______________________________________________
> boinc_dev mailing list
> [email protected]
> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.
>
>
>
_______________________________________________
boinc_dev mailing list
[email protected]
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.
