On Mon, May 20, 2013 at 12:35 AM, Varun Sharma <[email protected]> wrote: > Hi Lars, > > Thanks for the response. > > Regarding #2 again, so if RS1 failed, then the following happens... > 1) RS2 takes over its logs... > 2) Master renames the log containing directory to have a -splitting in the > path > 3) Does RS2 already know about the "-splitting" path ?
It will look at all the possible locations. See ReplicationSource.openReader > > Also on a related note, was there a reason that we have all region servers > watching all other region server's queue of logs. Otherwise, couldn't the > master have done the reassignment of outstanding logs to other region > servers more fairly upon failure ? I think I did it like that because it was easier since the region server has to be told to grab the queue(s) anyway. > > Thanks > Varun > > > On Sun, May 19, 2013 at 8:49 PM, lars hofhansl <[email protected]> wrote: > >> #1 yes >> #2 no >> >> :) >> >> Now, there are scenarios where inconsistencies can happen. The edits are >> not necessarily shipped in order when there are failures. >> So it is possible to have some Puts at T1 and some Deletes at T2 (T1 < >> T2), and end up with the deletes shipped first. >> Now imagine a compaction happens at the slave after the Deletes are >> shipped to the slave, but before the Puts are shipped... The Puts will >> reappear. >> >> -- Lars >> >> >> >> ________________________________ >> From: Varun Sharma <[email protected]> >> To: [email protected] >> Sent: Sunday, May 19, 2013 12:13 PM >> Subject: Questions about HBase replication >> >> >> Hi, >> >> I have a couple of questions about HBase replication... >> >> 1) When we ship edits to slave cluster - do we retain the timestamps in the >> edits - if we don't, I can imagine hitting some inconsistencies ? >> >> 2) When a region server fails, the master renames the directory containing >> WAL(s). Does this impact reading of those logs for replication ? >> >> Thanks >> Varun >>
