[
https://issues.apache.org/jira/browse/HBASE-14920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15217973#comment-15217973
]
Eshcar Hillel commented on HBASE-14920:
---------------------------------------
Hi,
[~stack] I will get back with answers to your questions/comments.
However, first I encountered an issue which I would like to consult about.
During in-memory compaction we write aside the minimum sequence id in the
segment, which is the result of the compaction.
We then use this sequence id to update the WAL sequence Id accounting so that
old WAL files can be discarded.
What I discovered is that there are cells the are not regular key-value cells,
which reside in a segment and have very small sequence id (e.g., 8). I guess
these cells are being added automatically by the store, and not as a result of
the client operation.
These cells fall through the compaction filter, so they always land on the
resulting segment which means the minimum sequence id of the segment is always
small and as a consequence WAL files are not discarded (and at some point may
exceed the limit on the number of WAL files - currently 33).
A possible solution would be to ignore these cells while doing the bookkeeping
of the minimal sequence id.
Namely, only consider cells of type Put or Delete when setting the minimum seq
id of the segment.
Does this seem a reasonable solution?
Here is a full list of possible types: Minimum, Put, Delete,
DeleteFamilyVersion, DeleteColumn, DeleteFamily Maximum.
> Compacting Memstore
> -------------------
>
> Key: HBASE-14920
> URL: https://issues.apache.org/jira/browse/HBASE-14920
> Project: HBase
> Issue Type: Sub-task
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Attachments: HBASE-14920-V01.patch, HBASE-14920-V02.patch,
> move.to.junit4.patch
>
>
> Implementation of a new compacting memstore with non-optimized immutable
> segment representation
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
