Philippe Verdy <> changed:

           What    |Removed                     |Added
                 CC|                            |

--- Comment #4 from Philippe Verdy <> ---
If there are huge cohorts, may be the tool could automatically split it into
subcohorts, then it will schedule each subcohort separately, and will create a
temporary report storing results that will then be aggregated.
It is opssible if the SQL queries contain only aggregatable items (all of them
should be aggregatable, because Wikimetrics should only be used to generate
aggregate data to respect users privacy)

So all data columns should specify the type aggregate they use: COUNT, MIN,

Derived aggregates can be computed in a scheduled way using only these basic
aggregates: this includes AVG (uses SUM and COUNT), STDDEV or VAR (uses
SUM(data), SUM(data^2) and COUNT).

The scheduler would then report the status of each subcohort processed and if
needed it can be paused at any time when it has already run for too long but
there are enough data generated to create a valid report, and resumed later
when the servers experiment lower work charges. The scheduler should also be
able to monitor the time or work charge taken by each subcohort, in order to
estimate and adjust the size of the next subcohort; or to insert variables
delay before processing the next subcohort.

An SQL server admin could also kill a SQL query that takes too much
time/resource: that query will fail, the scheduler will detect the failure and
pause the processing intil the cohort parameters are adjusted and the scheduler
being relaunched to restart the work for the last failed subcohort. This could
allow manual tuning of these cohort sizes.

(the cohort uploader may also consider splitting this cohort himself into
multiple ones with reasonnable size. The same cohort creator should not have
multiple cohorts being processed at the same time, but he could schedul them in

You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
Wikibugs-l mailing list

Reply via email to