in hedwig one hub does both the publish and subscribe for a given
topic and therefore is the only processes reading and writing from/to a
ledger, so there isn't an issue.
The ReadAheadCache does read-ahead :) it is so that we can minimize
latency when doing sequential reads.
ben
On 10/21/2010 11:30 PM, amit jaiswal wrote:
Hi,
How does Hedwig handles this scenario? Since only one of the hubs have the
ownership of a topic, the same hub is able to serve both publish and subscribe
requests concurrently. Is my understanding correct ?
Also, what is the purpose of ReadAheadCache class in Hedwig? Is it used
somewhere for this concurrent read/write problem?
-regards
Amit
----- Original Message ----
From: Benjamin Reed<br...@yahoo-inc.com>
To: zookeeper-user@hadoop.apache.org
Sent: Fri, 22 October, 2010 11:09:07 AM
Subject: Re: Is it possible to read/write a ledger concurrently
currently program1 can read and write to an open ledger, but program2 must wait
for the ledger to be closed before doing the read. the problem is that program2
needs to know the last valid entry in the ledger. (there may be entries that may
not yet be valid.) for performance reasons, only program1 knows the end. so you
need a way to propagate that information.
we have talked about a way to push the last entry into the bookkeeper handle.
flavio was working on it, but i don't think it has been implemented.
ben
On 10/21/2010 10:22 PM, amit jaiswal wrote:
Hi,
In BookKeeper documentation, the sample program creates a ledger, writes some
entries and then *closes* the ledger. Then a client program opens the ledger,
and reads the entries from it.
Is it possible for program1 to write to a ledger, and program2 to read from
the
ledger at the same time. In BookKeeper code, if a client tries to read from a
ledger which is not being closed (as per its metadata in zk), then a recovery
process is started to check for consistency.
Waiting for ledger to get closed can introduce lot of latency at the client
side. Can somebody explain this functionality?
-regards
Amit
