When you say that 'create a ledger and record the ledger id in some other places' is bad, I assume that you're referring to the number of metadata accesses. The pattern itself is not bad; BookKeeper provides a ledger abstraction, the application designer decides the best way of using those ledgers. Storing the ledgers elsewhere is a common pattern indeed, but again, managed ledgers have been proposed in my understanding to cover common patterns like this one.
About the use of names instead of longs, I believe one reason we did it is that we didn't initially expect users to manipulate ledgers directly, so programs manipulate the ledgers. Consequently, doing things like comparing two ledger ids is more efficient with longs. Granted that for administration perhaps human readable names could be easier. > Removing metadata accessing means > you could not change ledgers, a better way to do that is to 'shrink' a > ledger. I think this is a great point, but can we actually review why publishes are causing metadata accesses? This should only happen upon topic creation, acquisition, or when rolling the log, no? For the first one and for the particular application I believe we are discussing, we shouldn't be creating and destroying topics as frequently as we are. Topic acquisition is a problem in the case of crashes, but for topic acquisition, we'll need at least one access to the metadata anyway, although we could amortize an access across many ledgers if we have groups. For the last one, I believe we can increase the frequency of rolling. -Flavio On Jan 18, 2013, at 7:56 AM, Sijie Guo <[email protected]> wrote: > As I pointed, the style 'create a ledger and record the ledger id in some > other places' is bad, especially changing ledger would cause 2 metadata > writes and it is in the publish path, which caused latency spike (the > latency spike depends on the two metadata writes latency). The reason why I > proposed using 'ledger name', 're-open' and 'shrink', is trying to remove > any metadata accesses during publishes. Removing metadata accessing means > you could not change ledgers, a better way to do that is to 'shrink' a > ledger. > > -Sijie > > On Thu, Jan 17, 2013 at 7:26 AM, Ivan Kelly <[email protected]> wrote: > >> Also, I don't think the shrink operation is necessary. The aim here is >> to avoid the metadata spike, yes? If so, we can still roll the ledger >> when it gets to a certain capacity, and this is unlikely to happen on >> a spike. >>
