Since Jingcheng did the majority of the work, I'd like him to stat the thread vote thread. It should happen in the next few days.
Jon On Tue, Jul 14, 2015 at 3:53 PM, Andrew Purtell <[email protected]> wrote: > Start a vote thread? This says DISCUSSION > > On Tue, Jul 14, 2015 at 2:05 PM, Ted Yu <[email protected]> wrote: > > > There have been several iterations since the first mega patch was posted. > > > > Currently QA run is green. > > > > IntegrationTestIngestWithMOB has been run which passes. > > > > I want to give +1 for merging to master branch. > > > > On Tue, Jul 7, 2015 at 9:37 PM, Anoop John <[email protected]> > wrote: > > > > > +1 > > > > > > We will work on making it a Vote thread. > > > > > > -Anoop- > > > > > > On Wed, Jul 8, 2015 at 9:46 AM, ramkrishna vasudevan < > > > [email protected]> wrote: > > > > > > > +1 to start the voting again. > > > > > > > > On Wed, Jul 8, 2015 at 9:07 AM, Ted Yu <[email protected]> wrote: > > > > > > > > > Looks like all MOB-related JIRAs have been resolved. > > > > > > > > > > Should the voting process be resumed ? > > > > > > > > > > Cheers > > > > > > > > > > On Thu, May 28, 2015 at 5:48 PM, Anoop John <[email protected] > > > > > > wrote: > > > > > > > > > > > Yes Andy. The sweep tool is completely optional now. We have a > > chore > > > > > doing > > > > > > the compaction, like we trigger auto major compaction. We can > > > > configure > > > > > > the interval. Auto can be turned off and user can explicitly call > > > also. > > > > > We > > > > > > have shell and API support. > > > > > > > > > > > > Anoop > > > > > > > > > > > > On Friday, May 29, 2015, Andrew Purtell <[email protected]> > > wrote: > > > > > > > MOB references in cells won't find their value if the MOB hfile > > has > > > > > been > > > > > > > corrupted. Dealing with that would be like any other corrupted > > > HFile, > > > > > > > understood. The dangling references make thinking about > (partial) > > > > > > recovery > > > > > > > and repair interesting. > > > > > > > > > > > > > > > > > > > > > On Thursday, May 28, 2015, Jingcheng Du < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > >> Andrew Purtell wrote > > > > > > >> > HBCK can check and sideline dangling reference files. I > think > > of > > > > MOB > > > > > > >> files > > > > > > >> > as "core enough" auxiliary files that need some support. I > > > suppose > > > > > > unlike > > > > > > >> > reference files their presence or absence won't produce a > > region > > > > > open > > > > > > >> > failure, we would see dangling pointers later when tying to > > > > service > > > > > > >> > queries. (Yes?) Will that abort the RS? Pardon the ignorant > > > > > question, > > > > > > >> > normally I could check the code but I'm at the airport on a > > > phone. > > > > > > >> > > > > > > >> Thanks for comments! > > > > > > >> In mob, usually the reference cells are committed after the > mob > > > > files > > > > > > are > > > > > > >> done. I think it hardly happens that a reference cell cannot > > find > > > > its > > > > > > mob > > > > > > >> file. > > > > > > >> Even if there's a dangling reference cell, the RS won't be > > > aborted, > > > > a > > > > > > empty > > > > > > >> cell is returned instead. > > > > > > >> > > > > > > >> > > > > > > >> Andrew Purtell wrote > > > > > > >> > On that subject, I should file follow up issues for more > check > > > and > > > > > > repair > > > > > > >> > options for HFiles. We should be able to detect missing or > > > corrupt > > > > > > files > > > > > > >> > of > > > > > > >> > all variety: HFile, reference, MOB. This may require an > > > expensive > > > > > scan > > > > > > >> > over > > > > > > >> > lots of files, but this is like fsck full disk surface scans > > and > > > > > those > > > > > > >> > have > > > > > > >> > similar costs. Providing MR based tools is fine but we > should > > > have > > > > > > >> > multithreaded tools that can stand in if a MR runtime is not > > > > > > available. > > > > > > >> > Import, Export, VerifyReplication...all of these tools are > in > > a > > > > > > >> different, > > > > > > >> > lesser, class than integrity and repair tools, in my > opinion. > > > > Since > > > > > > MOB > > > > > > >> > will likely be merged into trunk by then I'll be sure to > > include > > > > > it. I > > > > > > >> > agree it's not fair to ask more of MOB then what we have now > > for > > > > > > HFile. > > > > > > >> > > > > > > >> To detect the corrupt files, some code are needed in the file > > > > checker > > > > > to > > > > > > >> check mob files after knowing it's a mob-enabled column. > > > > > > >> To detect the missing or dangling reference cells, I think we > > have > > > > to > > > > > do > > > > > > a > > > > > > >> full-table scan, like what is done now in HFile.main. > > > > > > >> We can do that. > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> -- > > > > > > >> View this message in context: > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-hbase.679495.n3.nabble.com/DISCUSSION-Merge-of-the-hbase-11339-mob-branch-into-master-tp4071644p4071911.html > > > > > > >> Sent from the HBase Developer mailing list archive at > > Nabble.com. > > > > > > >> > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best regards, > > > > > > > > > > > > > > - Andy > > > > > > > > > > > > > > Problems worthy of attack prove their worth by hitting back. - > > Piet > > > > > Hein > > > > > > > (via Tom White) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > -- // Jonathan Hsieh (shay) // HBase Tech Lead, Software Engineer, Cloudera // [email protected] // @jmhsieh
