> On 2010-11-11 06:23:59, Kannan Muthukkaruppan wrote: > > Last major should be tracked on per region+CF basis, rather than just a per > > region basis, right? > > > > Nicolas's & Amit's recent changes to compaction algorithm already makes it > > such that different column families within the same region could end up > > having their major compactions done at different times. [A minor which > > includes all files will implicitly get promoted to a major]. > > > > Is there some boolean state/flag in the HFile's meta data already that > > tells us if the file was the result of a major compaction or not? > > > > If so, isn't it sufficient to use the timestamp of the file which has the > > "major compaction" tag to determine when a particular region+CF was last > > major compacted? > >
Kannan, yes, there is a flag on end of HFile already which says if file is result of a major compaction and yes, it would make more sense to do this on a per-Store level (though up to this its been at the Region level). And thanks, your note reminds me that all is different since the new compaction code went in. I'm thinking the latter probably even 'fixes' the original issue. Lets see. Meantime, I think we should move this issue out of 0.90. We can then take the time with the implementation. - stack ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1089/#review1906 ----------------------------------------------------------- On 2010-11-10 17:04:39, Jonathan Gray wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > http://review.cloudera.org/r/1089/ > ----------------------------------------------------------- > > (Updated 2010-11-10 17:04:39) > > > Review request for hbase, stack and khemani. > > > Summary > ------- > > This is a somewhat misguided attempt. It's not done but it shows the fairly > simple change to the actual major compaction code. > > The hard part is: > > + long lastMajor = region.getRegionInfo().getRegionData().getLastMajor(); > > And the question is where to persist that. > > This patch adds a new class called HRegionData into HRegionInfo that contains > any number of key-value pairs of data that get persisted with the HRI. Not > really sure how I ended up there but this data seemed like an odd-man-out so > adding another field seemed weird. We also need some kind of versioning in > HRI so we can add stuff w/o migrating. There's some versioned stuff in > HRData. > > Just looking for some feedback / ideas. > > > This addresses bugs HBASE-2990 and HBASE-3083. > http://issues.apache.org/jira/browse/HBASE-2990 > http://issues.apache.org/jira/browse/HBASE-3083 > > > Diffs > ----- > > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > 1033780 > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1033780 > > Diff: http://review.cloudera.org/r/1089/diff > > > Testing > ------- > > > Thanks, > > Jonathan > >
