Re: Unbounded memory usage for WQ > AQ ?

2021-01-18 Thread Venkateswara Rao Jujjuri
On Mon, Jan 18, 2021 at 10:53 AM Sijie Guo wrote: > > One concern for me in this thread is case (3). I'd expect a client that > doesn't crash to not give up, and eventually replace the bookie if it is > unresponsive. > > The current implementation doesn't retry replacing a bookie if an entry is

Re: Unbounded memory usage for WQ > AQ ?

2021-01-18 Thread Sijie Guo
On Mon, Jan 18, 2021 at 10:18 AM Flavio Junqueira wrote: > >>> Regarding recovery reads, recovery read doesn't need to be > deterministic. > >>> For the entry with (b1->success, b2->NoSuchLedger, b3->NoSuchLedger), > >>> either including it or excluding it in the sealed ledger is correct > >>>

Re: Unbounded memory usage for WQ > AQ ?

2021-01-18 Thread Sijie Guo
> One concern for me in this thread is case (3). I'd expect a client that doesn't crash to not give up, and eventually replace the bookie if it is unresponsive. The current implementation doesn't retry replacing a bookie if an entry is already acknowledged (receiving AQ responses). It relies on

Re: Unbounded memory usage for WQ > AQ ?

2021-01-18 Thread Flavio Junqueira
>>> Regarding recovery reads, recovery read doesn't need to be deterministic. >>> For the entry with (b1->success, b2->NoSuchLedger, b3->NoSuchLedger), >>> either including it or excluding it in the sealed ledger is correct >>> behavior. The bookkeeper client guarantees that once a ledger is

Re: Unbounded memory usage for WQ > AQ ?

2021-01-18 Thread Sijie Guo
Jack, Thank you for your replies! That's good as there are not violations of bookkeeper protocol. Comments inline. On Mon, Jan 18, 2021 at 3:20 AM Jack Vanlightly wrote: > > Did you guys see any issues with the ledger auditor? > > > The active writer can't guarantee it writing entries to WQ

Re: Unbounded memory usage for WQ > AQ ?

2021-01-18 Thread Flavio Junqueira
In the scenario that WQ > AQ, a client acknowledges the add of an entry e to the application once it receives AQ bookie acks. Say now that the client is not able to write a copy of e to at least one bookie b, it could be because: 1- The client crashed before it is able to do it 2- Bookie b

Re: Unbounded memory usage for WQ > AQ ?

2021-01-18 Thread Jack Vanlightly
> Did you guys see any issues with the ledger auditor? > The active writer can't guarantee it writing entries to WQ because it can > crash during retrying adding entries to (WQ - AQ) bookies. The need to repair AQ replicated entries is clear and the auditor is one such strategy. Ivan has also