On Wed, Mar 9, 2016 at 1:06 PM, Charan Reddy G <[email protected]> wrote:
> Added inline responses > > On Tue, Mar 8, 2016 at 10:04 PM, Sijie Guo <[email protected]> wrote: > >> On Tue, Mar 8, 2016 at 6:51 PM, Venkateswara Rao Jujjuri < >> [email protected]> >> wrote: >> >> > Hi Charan, please see my response below. >> > >> > Sijie/Matteo/others can you please chime in too? >> > >> > On Wed, Mar 2, 2016 at 11:40 AM, Charan Reddy G < >> [email protected]> >> > wrote: >> > >> > > Hi, >> > > >> > > I've few questions related to BookKeeper internals and I think some of >> > them >> > > are issues. Can someone please clarify them >> > > >> > > 1) SortedLedgerStorage: in SortedLedgerStorage in getEntry method, why >> > does >> > > it first try to read from EntryLogger (persistence storage) instead of >> > > EntryMemTable, isn't it optimal to first attempt to read from memory >> > > instead of from persistence storage? Is it empirical decision to go >> other >> > > way round? >> > > >> > >> > Last time I brought this up with Sijie and he thinks this is not the >> case. >> > Charan, >> > can you please provide call flow here? >> > >> >> I need to check this to confirm. >> Charan: Please check SortedLedgerStorage.getEntry() method >> >> > >> > >> > > 2) Cookie: why IndexDirs is not part of Cookie? In generateCookie >> method >> > of >> > > Cookie class, we added JournalDir and LedgerDirs but not IndexDirs, >> is it >> > > ok to not to include IndexDirs in it? >> > > >> > >> > Index dirs doesn't contain actual data, we need to treat it like >> metadata. >> > So, I believe we must not include index files. >> > >> >> I think the indexDirs were added later after we introduced Cookie. so we >> forgot to add them to the Cookie. >> >> This should be added though. >> Charan: so new layoutVersion of Cookie? Should we create a new >> Bug/Feature/Story for it? >> >> > >> > >> > > 3) MetaFormat BookieShell Command: On executing 'metaformat' >> BookieShell >> > > command it is observed that it does everything as expected but it >> doesn't >> > > deletes UnderReplicatedLedgers list from the ZooKeeper. It looks like >> an >> > > issue to me. >> > >> >> Could you file a jira for it? >> Charan: created jira Bug - >> https://issues.apache.org/jira/browse/BOOKKEEPER-903 >> >> > > >> > > 4) length in LedgerEntry: when we read LedgerEntry using LedgerHandle >> > > readEntries method, the length of LedgerEntry instance is not >> > representing >> > > the actual length of the entry instead it represents the cumulative >> > length >> > > of the entry in Ledger (entrylength of n = actual length of entry n + >> > > entrylength of (n-1)). Is this what is it supposed to be? Am I doing >> > > something wrong here? >> > > >> > >> > That is what it is; Entry length = header length + payload (the actual >> > payload from client) >> > >> > Charan: JV you misunderstood what I mentioned. I meant, I'm observing > that > Entry length (of entry n) = HeaderLength (of entry n) + Payload (of entry > n) + EntryLength (of entry (n-1)). > Sijie, can you please comment on this. > The length in LedgerEntry is an accumulative length of the data until this entry. The entry is a byte array, the actual length of an entry is the bytes.length. > > > >> > > 5) AutoRecoveryMain: it seems it is configurable to enable/disable >> > > AutoRecoveryDaemon and it runs as daemon thread in Bookie/BookieServer >> > > process only if it is enabled and the other option option is to run >> > > separate AutoRecoveryMain Daemon service process in each node from CLI >> > > (from bookkeeper shell script). Are there pro and cons to go one way >> or >> > the >> > > other? >> >> >> We run AutoRecoveryMain as separated service in our mesos scheduler. >> Because it is a stateless service. >> Charan: Thanks Sijie for clarification. I presume you are running >> AutoRecoveryMain >> service in each Bookie node. >> >> > Is it ok to not to run AutoRecoveryMain daemon in any of the >> > > bookies? >> > > >> > >> > This is preferred to be enabled. Disable is only for backward >> > compatibility. We even talked >> > about it last time when we met Sijie and Matteo at dinner. We can change >> > config file to enable it by default. >> > I believe this must be enabled on all bookies. >> > >> > >> > > >> > > 6) updatecookie command and updateledgers command: It is observed >> after >> > > calling updatecookie command it will update the cookies both in ZK and >> > > local filesystem directories according to the new BookieAddress, but >> if >> > we >> > > try listbookies command after updatecookie command it still shows old >> > > bookie address, is it not supposed to get updated as part of >> updatecookie >> > > command? will it not cause discrepancies? Similarly after >> updateledgers >> > > command listbookies command still show older bookieaddress. >> > > >> > >> >> Could you list the steps how to reproduce this? >> Charan: >> > Lets assume that there is a ledger with ledgerid 'n' and its ensemble is > <bookie1>, <bookie2>, <bookie3>. now lets assume there is change in > <bookie1> address (it could be change in hostname or portno or change in > USE_HOST_NAME_AS_BOOKIE_ID config value). so we would run 'updateledgers' > shell command (./bookkeeper shell updateledgers --bookieId hostname) to > reflect the <newbookie1> address. As expected after executing > 'updateledgers' command, if we get metadata of that ledger (shell > ledgermetadata -ledgerid <n>) then as we expected we would see <newbookie1> > in its ensemble, but when we get listbookies (shell listbookies -rw) it > still shows the old bookie address <bookie1> instead of <newbookie1>. > > Also by calling 'updatecookie' command in the case of change in > bookieaddress, cookie is getting updated both in ZK and locally with > <newbookie1> address, but 'listbookies' command still shows old address. > >> >> >> > >> > May be a bug?? >> > >> > >> > > >> > > Thanks, >> > > Charan Reddy >> > > >> > >> > >> > >> > -- >> > Jvrao >> > --- >> > First they ignore you, then they laugh at you, then they fight you, then >> > you win. - Mahatma Gandhi >> > >> > >
