On Fri, Jan 18, 2013 at 8:06 AM, Ivan Kelly <[email protected]> wrote:
> You touch on a key point here (it's in the the publish path). But it > doesn't have to be. There's two cases where this is in the publish > path, first publish and rolling the ledger. > > For the first publish to a topic, a ledger only needed to be created > if we are persisting the message, i.e. there are already subscribers > for that topic. If there's no subscribers, we can just discard the > message. > in most of case, there are subscriptions, since Hedwig's use case is reliable pub/sub system. so most of applications subscribed and never unsubscribe. so the case you described 'If there's no subscribers, we can just discard the message' is rare in Hedwig. > > In the second case, for rolling, we need to create a new ledger and > write it to the topic persistence info. This doesn't need to be in the > critical path though. They can occur after a message has been > persisted, to avoid the hit. > It affects the incoming publish requests at the 'ledger change' time period. Especially, for a continuous publish stream, how you decide the point to avoid the hit? One possible way is Hub server could pre-create ledger before the point changing ledger. pre-creation could run in background, not affecting publish path. but pre-creation increase the possibility to have zombie ledgers and exhausted ledger id space. And also it would make ledger change logic pretty complex. so it is not good.
