On Thu, Sep 27, 2012 at 3:16 AM, Ivan Kelly <[email protected]> wrote:
> > I took at look at the leveldb homepage and it says - "Only a single > process > > (possibly multi-threaded) can access a particular database at a time." > > This is bad because it means we can't run the console or any recovery > > related operations while the bookies are running. I may be wrong, though. > > What I had in mind was pretty simple. When we flush to the log file, we > > should simply flush entries sorted by the ledger id as key. Some changes > > might be needed to the ledger index cache, but I'm not very sure what the > > changes would be. What do you think? > This should be a fairly straightforward change to make, but would > require a bit of hacking around in the EntryLogger. Currently, when we > write to the entrylogger we write to the file handle and return, and > then the sync thread forces this to disk when it runs. Everything is > in OS buffers up until the sync. To do this sorting, we would need to > keep our own buffers and then both write to the channel _and_ force to > disk during the sync. > Yes, that is exactly what I had in mind. > > We could also cut down the amount of data we need to write to the > index files with this, we really only need to store the first entry > for the particular ledger in the that sync window, rather than an > index for every single entry. > Agreed. > > -Ivan >
