> On Oct. 16, 2016, 10:43 p.m., Anoop Sam John wrote: > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HeapMemStoreLAB.java, > > line 179 > > <https://reviews.apache.org/r/51785/diff/7-8/?file=1511963#file1511963line179> > > > > Able to get what u r trying to do here. Seems very complex now. > > So can we take a diff path? > > We have to merge N segments into one here means we have to merge N > > MSLABs referred by these segments. What we do as per this patch is make a > > new HeapMSLAB object and add all pooled chunk from other MSLABs int the Q > > in new one. And other stuff of refer next etc. > > > > Can we make a new MSLAB impl? This will act as a collection of N > > HeapMSLABs. Now we have to impl 4 methods > > incScannerCount and decScannerCount based on an openScannerCount in the > > new class. > > close() method like in HeapMSLAB and has to call close() on each of the > > N MSLABs it wrap > > This is an ImmutableSegment so no one is going to call copyCellInto(). > > Make this impl in this new class throw IllegalStateException or so. Will > > that look good? > > Michael Stack wrote: > No copying? That sounds good.
So this fix and so this comment is applicable to merge op alone.. There is still an open point which Me/Ram are not agreeing around this.. That is whether to do this in mem merge. This is required to limit the #segments it pipeline and to get bigger sized segment at tail of the pipeline. As we do flush of the tail of the segment. But our point is let us flush all segments (I mean flush to disk).. In case of in mem compaction (This action is by def OFF and as per data usage one can enable) it is worth to flush only tail. In this normal use case let us flush all segments and so there is only flatten of segments comes in and so no bug as this. There is absolutely no gain we get because of delaying of flush of the other segments in the pipeline ( I mean in merge kind of usages).. So I suggest let us discuss on that before going to work on this. Else it wl b waste of time. - Anoop Sam ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51785/#review152809 ----------------------------------------------------------- On Oct. 14, 2016, 12:49 p.m., Anastasia Braginsky wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51785/ > ----------------------------------------------------------- > > (Updated Oct. 14, 2016, 12:49 p.m.) > > > Review request for hbase. > > > Repository: hbase-git > > > Description > ------- > > This is a step toward final compacting memstore that allowes two modes of > work: index-compaction and data-compaction. > > The index-compaction means that when the new segment is pushed into the > pipeline, it is flattened and probably merged with old segments in the > pipeline. The new merge "feature" induces no data-copy-compaction and no > speculative SQM scan. > The compacting memstore of the data-compaction type means the usage of the > data-copy-compaction. > > > Diffs > ----- > > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java > 177f222 > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionPipeline.java > 6a13f43 > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HeapMemStoreLAB.java > 378601d > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ImmutableSegment.java > 12b7916 > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactor.java > 714ffe3 > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactorIterator.java > 9798ec2 > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactorSegmentsIterator.java > PRE-CREATION > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreMergerSegmentsIterator.java > PRE-CREATION > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreSegmentsIterator.java > PRE-CREATION > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SegmentFactory.java > 510ebbd > > hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/VersionedSegmentsList.java > 2e8bead > > hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingMemStore.java > 211a6d8 > > hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellArrayMapMemStore.java > f89a040 > > hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStoreLAB.java > 1ea5112 > > hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestPerColumnFamilyFlush.java > 6bfaa59 > > hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWalAndCompactingMemStoreFlush.java > 74826b0 > > Diff: https://reviews.apache.org/r/51785/diff/ > > > Testing > ------- > > > Thanks, > > Anastasia Braginsky > >
