Looks like all MOB-related JIRAs have been resolved. Should the voting process be resumed ?
Cheers On Thu, May 28, 2015 at 5:48 PM, Anoop John <[email protected]> wrote: > Yes Andy. The sweep tool is completely optional now. We have a chore doing > the compaction, like we trigger auto major compaction. We can configure > the interval. Auto can be turned off and user can explicitly call also. We > have shell and API support. > > Anoop > > On Friday, May 29, 2015, Andrew Purtell <[email protected]> wrote: > > MOB references in cells won't find their value if the MOB hfile has been > > corrupted. Dealing with that would be like any other corrupted HFile, > > understood. The dangling references make thinking about (partial) > recovery > > and repair interesting. > > > > > > On Thursday, May 28, 2015, Jingcheng Du <[email protected]> wrote: > > > >> Andrew Purtell wrote > >> > HBCK can check and sideline dangling reference files. I think of MOB > >> files > >> > as "core enough" auxiliary files that need some support. I suppose > unlike > >> > reference files their presence or absence won't produce a region open > >> > failure, we would see dangling pointers later when tying to service > >> > queries. (Yes?) Will that abort the RS? Pardon the ignorant question, > >> > normally I could check the code but I'm at the airport on a phone. > >> > >> Thanks for comments! > >> In mob, usually the reference cells are committed after the mob files > are > >> done. I think it hardly happens that a reference cell cannot find its > mob > >> file. > >> Even if there's a dangling reference cell, the RS won't be aborted, a > empty > >> cell is returned instead. > >> > >> > >> Andrew Purtell wrote > >> > On that subject, I should file follow up issues for more check and > repair > >> > options for HFiles. We should be able to detect missing or corrupt > files > >> > of > >> > all variety: HFile, reference, MOB. This may require an expensive scan > >> > over > >> > lots of files, but this is like fsck full disk surface scans and those > >> > have > >> > similar costs. Providing MR based tools is fine but we should have > >> > multithreaded tools that can stand in if a MR runtime is not > available. > >> > Import, Export, VerifyReplication...all of these tools are in a > >> different, > >> > lesser, class than integrity and repair tools, in my opinion. Since > MOB > >> > will likely be merged into trunk by then I'll be sure to include it. I > >> > agree it's not fair to ask more of MOB then what we have now for > HFile. > >> > >> To detect the corrupt files, some code are needed in the file checker to > >> check mob files after knowing it's a mob-enabled column. > >> To detect the missing or dangling reference cells, I think we have to do > a > >> full-table scan, like what is done now in HFile.main. > >> We can do that. > >> > >> > >> > >> -- > >> View this message in context: > >> > > http://apache-hbase.679495.n3.nabble.com/DISCUSSION-Merge-of-the-hbase-11339-mob-branch-into-master-tp4071644p4071911.html > >> Sent from the HBase Developer mailing list archive at Nabble.com. > >> > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > > >
