chewbranca commented on issue #4650: URL: https://github.com/apache/couchdb/issues/4650#issuecomment-1599213702
It's also worth remembering that the Folsom histograms are front loaded on the number of samples actually collected (for better or worse) which means that the percentage of couch_stats histogram updates that actually triggers an ets update goes down as the number of requests per second goes up. I personally think this is not great as the default bucket size of 1024 entries in a given one second "moment", leaves huge amounts of data on the floor; for example, if 100k histogram updates are called in a given second, the last update will have something like a 1:99k chance of actually impacting the histogram. So one thing to watch out for is that switching to a bucketing based histogram that is explicitly accurate in incrementing a bucket for every histogram update, we will drastically increase the amount of lock contention happening. In our example above, all 100k requests will actually modify the histogram, as opposed to Folsom's sampling based approach which has a chance to update the histogram inversely proportional to the number of requests made in the current moment. So again, I believe the fundamental issue at hand is concurrent updates to histograms, and any implementation we run with needs to address that issue directly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
