On 02/03/17 09:52 AM, Julien Danjou wrote: >> using hashring idea, the buckets will be distributed among all the >> > active metricd agents. the metricd agents will loop through all the >> > assigned buckets based on processing interval. the actual processing of >> > each bucket will be similar to what we have now: grab metrics, queue it >> > for processing workers. the only difference is instead of just grabbing >> > first x metrics and stopping, we keep grabbing until bucket is 'clear'. >> > this will help us avoid the current issue where some metrics are never >> > scheduled because the return order puts it at the end. > It does not change that much the current issue IIUC. The only difference > is that now we have 1 bucket and N metricd trying to empty it, whereas > now we would have M buckets with N metricd trying spread so there's M/N > metricd per bucket trying to empty each bucket. :) > > At some scale (larger than currently) it will improve things but it does > not seem to be a drastic change. > > (I am also not saying that I have a better solution :) >
one of the issues we can't effectively partition the single bucket is partly because we have multiple agents on a single bucket. in theory we can use markers to partition the single bucket but because of multiple workers, the marker has a very high chance of disappearing. in this case, only one agents is ever working on a bucket so it should minimise the chance of a marker disappearing and therefore let us go deeper into bucket. -- gord __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev