That's great, and I also appreciate the time spent discussing on this thread. I plan to vote yes.
On Tue, Jul 14, 2015 at 3:57 PM, Jonathan Hsieh <[email protected]> wrote: > Since Jingcheng did the majority of the work, I'd like him to stat the > thread vote thread. It should happen in the next few days. > > Jon > > On Tue, Jul 14, 2015 at 3:53 PM, Andrew Purtell <[email protected]> > wrote: > > > Start a vote thread? This says DISCUSSION > > > > On Tue, Jul 14, 2015 at 2:05 PM, Ted Yu <[email protected]> wrote: > > > > > There have been several iterations since the first mega patch was > posted. > > > > > > Currently QA run is green. > > > > > > IntegrationTestIngestWithMOB has been run which passes. > > > > > > I want to give +1 for merging to master branch. > > > > > > On Tue, Jul 7, 2015 at 9:37 PM, Anoop John <[email protected]> > > wrote: > > > > > > > +1 > > > > > > > > We will work on making it a Vote thread. > > > > > > > > -Anoop- > > > > > > > > On Wed, Jul 8, 2015 at 9:46 AM, ramkrishna vasudevan < > > > > [email protected]> wrote: > > > > > > > > > +1 to start the voting again. > > > > > > > > > > On Wed, Jul 8, 2015 at 9:07 AM, Ted Yu <[email protected]> > wrote: > > > > > > > > > > > Looks like all MOB-related JIRAs have been resolved. > > > > > > > > > > > > Should the voting process be resumed ? > > > > > > > > > > > > Cheers > > > > > > > > > > > > On Thu, May 28, 2015 at 5:48 PM, Anoop John < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > Yes Andy. The sweep tool is completely optional now. We have a > > > chore > > > > > > doing > > > > > > > the compaction, like we trigger auto major compaction. We can > > > > > configure > > > > > > > the interval. Auto can be turned off and user can explicitly > call > > > > also. > > > > > > We > > > > > > > have shell and API support. > > > > > > > > > > > > > > Anoop > > > > > > > > > > > > > > On Friday, May 29, 2015, Andrew Purtell <[email protected]> > > > wrote: > > > > > > > > MOB references in cells won't find their value if the MOB > hfile > > > has > > > > > > been > > > > > > > > corrupted. Dealing with that would be like any other > corrupted > > > > HFile, > > > > > > > > understood. The dangling references make thinking about > > (partial) > > > > > > > recovery > > > > > > > > and repair interesting. > > > > > > > > > > > > > > > > > > > > > > > > On Thursday, May 28, 2015, Jingcheng Du < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > >> Andrew Purtell wrote > > > > > > > >> > HBCK can check and sideline dangling reference files. I > > think > > > of > > > > > MOB > > > > > > > >> files > > > > > > > >> > as "core enough" auxiliary files that need some support. I > > > > suppose > > > > > > > unlike > > > > > > > >> > reference files their presence or absence won't produce a > > > region > > > > > > open > > > > > > > >> > failure, we would see dangling pointers later when tying > to > > > > > service > > > > > > > >> > queries. (Yes?) Will that abort the RS? Pardon the > ignorant > > > > > > question, > > > > > > > >> > normally I could check the code but I'm at the airport on > a > > > > phone. > > > > > > > >> > > > > > > > >> Thanks for comments! > > > > > > > >> In mob, usually the reference cells are committed after the > > mob > > > > > files > > > > > > > are > > > > > > > >> done. I think it hardly happens that a reference cell cannot > > > find > > > > > its > > > > > > > mob > > > > > > > >> file. > > > > > > > >> Even if there's a dangling reference cell, the RS won't be > > > > aborted, > > > > > a > > > > > > > empty > > > > > > > >> cell is returned instead. > > > > > > > >> > > > > > > > >> > > > > > > > >> Andrew Purtell wrote > > > > > > > >> > On that subject, I should file follow up issues for more > > check > > > > and > > > > > > > repair > > > > > > > >> > options for HFiles. We should be able to detect missing or > > > > corrupt > > > > > > > files > > > > > > > >> > of > > > > > > > >> > all variety: HFile, reference, MOB. This may require an > > > > expensive > > > > > > scan > > > > > > > >> > over > > > > > > > >> > lots of files, but this is like fsck full disk surface > scans > > > and > > > > > > those > > > > > > > >> > have > > > > > > > >> > similar costs. Providing MR based tools is fine but we > > should > > > > have > > > > > > > >> > multithreaded tools that can stand in if a MR runtime is > not > > > > > > > available. > > > > > > > >> > Import, Export, VerifyReplication...all of these tools are > > in > > > a > > > > > > > >> different, > > > > > > > >> > lesser, class than integrity and repair tools, in my > > opinion. > > > > > Since > > > > > > > MOB > > > > > > > >> > will likely be merged into trunk by then I'll be sure to > > > include > > > > > > it. I > > > > > > > >> > agree it's not fair to ask more of MOB then what we have > now > > > for > > > > > > > HFile. > > > > > > > >> > > > > > > > >> To detect the corrupt files, some code are needed in the > file > > > > > checker > > > > > > to > > > > > > > >> check mob files after knowing it's a mob-enabled column. > > > > > > > >> To detect the missing or dangling reference cells, I think > we > > > have > > > > > to > > > > > > do > > > > > > > a > > > > > > > >> full-table scan, like what is done now in HFile.main. > > > > > > > >> We can do that. > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> -- > > > > > > > >> View this message in context: > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-hbase.679495.n3.nabble.com/DISCUSSION-Merge-of-the-hbase-11339-mob-branch-into-master-tp4071644p4071911.html > > > > > > > >> Sent from the HBase Developer mailing list archive at > > > Nabble.com. > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Best regards, > > > > > > > > > > > > > > > > - Andy > > > > > > > > > > > > > > > > Problems worthy of attack prove their worth by hitting back. > - > > > Piet > > > > > > Hein > > > > > > > > (via Tom White) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > > > > > > -- > // Jonathan Hsieh (shay) > // HBase Tech Lead, Software Engineer, Cloudera > // [email protected] // @jmhsieh > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
