Added inline responses On Tue, Mar 8, 2016 at 10:04 PM, Sijie Guo <[email protected]> wrote:
> On Tue, Mar 8, 2016 at 6:51 PM, Venkateswara Rao Jujjuri < > [email protected]> > wrote: > > > Hi Charan, please see my response below. > > > > Sijie/Matteo/others can you please chime in too? > > > > On Wed, Mar 2, 2016 at 11:40 AM, Charan Reddy G <[email protected] > > > > wrote: > > > > > Hi, > > > > > > I've few questions related to BookKeeper internals and I think some of > > them > > > are issues. Can someone please clarify them > > > > > > 1) SortedLedgerStorage: in SortedLedgerStorage in getEntry method, why > > does > > > it first try to read from EntryLogger (persistence storage) instead of > > > EntryMemTable, isn't it optimal to first attempt to read from memory > > > instead of from persistence storage? Is it empirical decision to go > other > > > way round? > > > > > > > Last time I brought this up with Sijie and he thinks this is not the > case. > > Charan, > > can you please provide call flow here? > > > > I need to check this to confirm. > Charan: Please check SortedLedgerStorage.getEntry() method > > > > > > > > 2) Cookie: why IndexDirs is not part of Cookie? In generateCookie > method > > of > > > Cookie class, we added JournalDir and LedgerDirs but not IndexDirs, is > it > > > ok to not to include IndexDirs in it? > > > > > > > Index dirs doesn't contain actual data, we need to treat it like > metadata. > > So, I believe we must not include index files. > > > > I think the indexDirs were added later after we introduced Cookie. so we > forgot to add them to the Cookie. > > This should be added though. > Charan: so new layoutVersion of Cookie? Should we create a new > Bug/Feature/Story for it? > > > > > > > > 3) MetaFormat BookieShell Command: On executing 'metaformat' > BookieShell > > > command it is observed that it does everything as expected but it > doesn't > > > deletes UnderReplicatedLedgers list from the ZooKeeper. It looks like > an > > > issue to me. > > > > Could you file a jira for it? > Charan: created jira Bug - > https://issues.apache.org/jira/browse/BOOKKEEPER-903 > > > > > > > 4) length in LedgerEntry: when we read LedgerEntry using LedgerHandle > > > readEntries method, the length of LedgerEntry instance is not > > representing > > > the actual length of the entry instead it represents the cumulative > > length > > > of the entry in Ledger (entrylength of n = actual length of entry n + > > > entrylength of (n-1)). Is this what is it supposed to be? Am I doing > > > something wrong here? > > > > > > > That is what it is; Entry length = header length + payload (the actual > > payload from client) > > > Charan: JV you misunderstood what I mentioned. I meant, I'm observing that Entry length (of entry n) = HeaderLength (of entry n) + Payload (of entry n) + EntryLength (of entry (n-1)). Sijie, can you please comment on this. > > > > 5) AutoRecoveryMain: it seems it is configurable to enable/disable > > > AutoRecoveryDaemon and it runs as daemon thread in Bookie/BookieServer > > > process only if it is enabled and the other option option is to run > > > separate AutoRecoveryMain Daemon service process in each node from CLI > > > (from bookkeeper shell script). Are there pro and cons to go one way or > > the > > > other? > > > We run AutoRecoveryMain as separated service in our mesos scheduler. > Because it is a stateless service. > Charan: Thanks Sijie for clarification. I presume you are running > AutoRecoveryMain > service in each Bookie node. > > > Is it ok to not to run AutoRecoveryMain daemon in any of the > > > bookies? > > > > > > > This is preferred to be enabled. Disable is only for backward > > compatibility. We even talked > > about it last time when we met Sijie and Matteo at dinner. We can change > > config file to enable it by default. > > I believe this must be enabled on all bookies. > > > > > > > > > > 6) updatecookie command and updateledgers command: It is observed after > > > calling updatecookie command it will update the cookies both in ZK and > > > local filesystem directories according to the new BookieAddress, but if > > we > > > try listbookies command after updatecookie command it still shows old > > > bookie address, is it not supposed to get updated as part of > updatecookie > > > command? will it not cause discrepancies? Similarly after updateledgers > > > command listbookies command still show older bookieaddress. > > > > > > > Could you list the steps how to reproduce this? > Charan: > Lets assume that there is a ledger with ledgerid 'n' and its ensemble is <bookie1>, <bookie2>, <bookie3>. now lets assume there is change in <bookie1> address (it could be change in hostname or portno or change in USE_HOST_NAME_AS_BOOKIE_ID config value). so we would run 'updateledgers' shell command (./bookkeeper shell updateledgers --bookieId hostname) to reflect the <newbookie1> address. As expected after executing 'updateledgers' command, if we get metadata of that ledger (shell ledgermetadata -ledgerid <n>) then as we expected we would see <newbookie1> in its ensemble, but when we get listbookies (shell listbookies -rw) it still shows the old bookie address <bookie1> instead of <newbookie1>. Also by calling 'updatecookie' command in the case of change in bookieaddress, cookie is getting updated both in ZK and locally with <newbookie1> address, but 'listbookies' command still shows old address. > > > > > > May be a bug?? > > > > > > > > > > Thanks, > > > Charan Reddy > > > > > > > > > > > -- > > Jvrao > > --- > > First they ignore you, then they laugh at you, then they fight you, then > > you win. - Mahatma Gandhi > > >
