On Tue, Sep 25, 2012 at 1:35 AM, Flavio Junqueira <[email protected]> wrote:
> Just to add a couple of comments to the discussion, separating reads and > writes into different threads should only help with queuing latency. It > wouldn't help with IO latency. > Yes, but with the current implementation, publishes latencies in hedwig suffer because of lagging subscribers. By separating read and write queues, we can at least guarantee that the write SLA is maintained (separate journal disk + separate thread would ensure that writes are not affected by read related seeks) > > Also, it sounds like a good idea to have at least one thread per ledger > device. In the case of multiple ledger devices, if we use one single > thread, then the performance of the bookie will be driven by the slowest > disk, no? > yup, makes sense. > > -Flavio > > On Sep 25, 2012, at 10:24 AM, Ivan Kelly wrote: > > >> Could you give some information on what those shortcomings are? Also, do > >> let me know if you need any more information from our end. > > Off the top of my head: > > - reads and writes are handled in the same thread (as you have observed) > > - each entry read requires a single RPC. > > - entries are read in parallel > By parallel, you mean the BufferedChannel wrapper on top of FileChannel, right? > > > > Not all of these could result in the high latency you see, but if each > > entry is being read separately, a sync on the ledger disk in between > > will make a mess of the disk head scheduling. > Increasing the time interval between flushing log files might possibly help in this case then? > > > > -Ivan > > Thanks for the help :)
