Could you elaborate a bit more on : “ The Bookie handles both graceful stops and machine crashes without need of a manual operation. “
When a SIGTERM is issued to a bookie process, does any “shutdown” method get invoked that would take care of doing the “data flush” as mentioned here: https://bookkeeper.apache.org/docs/4.7.3/getting-started/concepts/#data-flush Specifically : The bookie needs to ensure that the entry log flushes its buffered data during shutdown. Otherwise, entry log files become corrupted with partial entries. I was wondering what is the way to make sure the entry log data is flushed before the bookie is shut down? What could we check in the bookie logs that tells us that the shutdown process was initiated after SIGTERM and entry log flush has completed. We would consider deleting the bookie pod after this. -Thanks, Prajakta From: Enrico Olivelli <eolive...@gmail.com> Sent: Tuesday, December 3, 2019 5:34 PM To: user Cc: Flavio Junqueira; Sharda, Ravi Subject: Re: Bookie graceful shutdown and restart [EXTERNAL EMAIL] Il giorno mar 3 dic 2019 alle ore 11:41 <prajakta.belgu...@dell.com<mailto:prajakta.belgu...@dell.com>> ha scritto: Hi Bookkeeper Team, We’re currently running into an issue where on bookie restart it is not able to find certain entries in ledgers and hence reads for these entries keep failing. In most cases, the bookie is able to recover from these errors after a period and things work as usual. However, in a case where all bookies are deleted simultaneously, the recovery never happens and we’re stuck in a state were we see data loss. I needed to understand what is the guidance to ensure graceful termination of a bookie? What is the right way to handle bookie restarts? The Bookie handles both graceful stops and machine crashes without need of a manual operation. Is there an option to move a bookie to a “readonly” mode prior to shutdown so we can sort of switch off writes? This is done automatically in recent BookKeeper versions (IIRC from 4.8+) Is auto recovery supposed to correct the missing data in a bookie post restart? I suppose you run into some kind of bug, but autorecovery will write again the missing entries on the Bookie, so it should fix the problem. a copy of the entry must be available on another Bookie (up and running) In this case the solve could be to run manual recovery every time a bookie restarts.. Enrico -Thanks, Prajakta
