On Fri, Aug 4, 2017 at 8:50 AM, Enrico Olivelli <[email protected]> wrote:
> 2017-08-04 17:38 GMT+02:00 Sijie Guo <[email protected]>: > > > On Aug 4, 2017 5:27 AM, "Enrico Olivelli" <[email protected]> wrote: > > > > Hi bookkeepers, > > I noticed that digests are computed on the mailWorkerPool during addEntry > > I think that it is better to do such computation outside the pool > > > > > > I have drafted a patch to share my code > > https://github.com/apache/bookkeeper/pull/389 > > the patch covers only LedgerHandleAdv as it is just a proof of concept > > > > opinions ? > > > > > > Because there is always a thread need to run this computation, I am not > > sure how much performance gain you can get from this? > > > > Do you have any performance results before making any such changes? > > > > not yet > > I see your point: as the number of CPUs is limited from an higher point of > view the global throughput will not change > > My idea comes from the fact the addEntry is called from applications > threads, the mainWorkerPool is used for delivering callbacks, if I keep > busy a thread from that pool and it is bounded in size I am limiting the > capacity to deliver callbacks to application threads which are waiting for > acks of adds or reads. > But if you are looking from end-to-end perspective (from you issue the request to receiving the callback), there isn't really difference. Because the digest computation needs to run in some threads, your throughput and latency will still be bounded. Especially if you think the problem from other services (e.g. pulsar brokers, dl proxies), I doubt it will give you any performance improvement. > > Another point is that it easy to track them/profile "application" threads, > the mainWorkerPool is shared and does many kind of operations. > It might be `easy` for a specific application, but not really `easy` for a lot of applications. Especially for the applications that writes a lot of ledgers. > > I will do some tests and share my results (as soon as I will be back from > vacation) > Yes. Please show the numbers, and please cover as many use cases as possible. > > -- Enrico > > > > > > > Sijie > > > > > > -- Enrico > > >
