[
https://issues.apache.org/jira/browse/HBASE-14920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15216720#comment-15216720
]
stack commented on HBASE-14920:
-------------------------------
I like the notion of a 'CompactionMemStore' instead of a plain old MemStore.
Reading the nice description made me think of how we need to work on recovery.
There'll be more to recover if we can carry more in memory. Another issue.
This needs a comment on it: IN_MEMORY_FLUSH_THRESHOLD_FACTOR
Would be good if could use the Interface Store rather than implementation
HStore in below
66 private HStore store;
For these...
// A flag for tests only
... we normally just flag with @VisibleForTesting annotation. FYI.
I've asked this before but forgot the answer, this has to be here:
isCompactingMemStore ?
Is this a race in the below?
184 @Override public long getFlushableSize() {
185 long snapshotSize = getSnapshot().getSize();
186 if(snapshotSize == 0) {
187 //if snapshot is empty the tail of the pipeline is flushed
188 snapshotSize = pipeline.getTailSize();
189 }
190 return snapshotSize > 0 ? snapshotSize : keySize();
191 }
If no snapshot, use the tail of the pipeline? The pipeline tail has not yet
been moved to be the new snapshot?
If sequenceid is MAX, don't we want to update it?
if(minSequenceId != Long.MAX_VALUE) {
I asked this before too...so, to do compaction, we need to have exclusive lock
or exclusive update lock just while you swap segments (the compacted for the
inputs?)... it looks like the next method, flushInMemory, answers my question
so ignore..
245 /* The thread is dispatched to flush-in-memory. This cannot be
done
246 * on the same thread, because for flush-in-memory we require
updatesLock
247 * in exclusive mode while this method (checkActiveSize) is
invoked holding updatesLock
248 * in the shared mode. */
Any guard against queuing same old compactions over and over again?
250 if(pool != null) { // the pool can be null in some tests
scenarios
251 InMemoryFlushWorker worker = new InMemoryFlushWorker();
252 LOG.info("Dispatching the MemStore in-memory flush for store "
253 + store.getColumnFamilyName());
254 pool.submit(worker);
255 inMemoryFlushInProgress.set(true);
256 }
... say there is a bug or we are slow to run already-queued compactions?
s/stopCompact/stopCompact/g
The below would seem to say we compact with updates lock held. Fix if wrong...
if it is true, then lets discuss:
351 * It takes the updatesLock exclusively, pushes active into the
pipeline,
352 * and compacts the pipeline.
Got as far as MemstoreCompactor... will be back.
> Compacting Memstore
> -------------------
>
> Key: HBASE-14920
> URL: https://issues.apache.org/jira/browse/HBASE-14920
> Project: HBase
> Issue Type: Sub-task
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Attachments: HBASE-14920-V01.patch, HBASE-14920-V02.patch
>
>
> Implementation of a new compacting memstore with non-optimized immutable
> segment representation
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
