On Thu, Jan 17, 2013 at 10:56:12PM -0800, Sijie Guo wrote: > 'create a ledger and record the ledger id in some other places' is bad I disagree. This approach allows us to keep the bookkeeper interface completely decoupled from whatever is using it. It keeps the data model of bookkeeper simple.
> especially changing ledger would cause 2 metadata writes and it is > in the publish path, which caused latency spike (the latency spike > depends on the two metadata writes latency). You touch on a key point here (it's in the the publish path). But it doesn't have to be. There's two cases where this is in the publish path, first publish and rolling the ledger. For the first publish to a topic, a ledger only needed to be created if we are persisting the message, i.e. there are already subscribers for that topic. If there's no subscribers, we can just discard the message. In the second case, for rolling, we need to create a new ledger and write it to the topic persistence info. This doesn't need to be in the critical path though. They can occur after a message has been persisted, to avoid the hit. > The reason why I proposed using 'ledger name', 're-open' and > 'shrink', is trying to remove any metadata accesses during > publishes. Removing metadata accessing means you could not change > ledgers, a better way to do that is to 'shrink' a ledger. I think this can be achieved with the 're-open' operation alone. The reason I'm being so resistent to 'ledger name' and 'shrink', is that I don't like changing lower layers until it's absolutely sure that nothing can be done at the higher layers to solve the problem, i.e. when it shows that there's a fundamental shortcoming in the lower layer. I don't think that is the case for ledger name and shrink. -Ivan
