[
https://issues.apache.org/jira/browse/HBASE-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891285#action_12891285
]
HBase Review Board commented on HBASE-2863:
-------------------------------------------
Message from: "Ryan Rawson" <[email protected]>
bq. On 2010-07-22 06:57:10, stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java, line
384
bq. > <http://review.hbase.org/r/354/diff/1/?file=3014#file3014line384>
bq. >
bq. > Could this first kv on the row in memstore have a ts in advance of
'now'? I suppose it can't -- least it shouldn't be possible, right?
bq. >
bq. > If thousands of updates a second, could this be a prob? This logic?
so what was happening previously:
- snapshot happens
- we do an increment which found there was no 'existing' TS in memstore to
'update' so we make a new one with 'now'.
- we end up with two KVs, one in snapshot one in memstore both with the same
timestamp.
Another issue (which the unit test perfectly tests for) is so:
- time=1 snapshot occurs with a KV ts=1 into snapshot
- time=1 ICV happens, but now the KV in memstore _must_ have a TS=2 (or else we
get duplicate TS - bad!)
- time=1 ICV happens, but the KV in memstore is TS=2, but now=1, so we need to
keep the max(now,TS in memstore) or else we get potential duplicates
In both these cases if we have a Put with ts=Now+X we have problems:
- in HRegion we do a Get and we see Value=100 ts=Now+X
- in Store we do updateColumnValue with Value=101, TS=now
- If we dont clear that ts=Now+X we will put a shadowed KV, and the _next_ ICV
will see value '100' not value '101' and we will never actually 'increment'
until we get past Now+X. The compromise is to clear out all KVs in memstore
period.
We can end up with a situation where the KV in memstore leads 'real time' by a
millisecond during snapshots. If we had a way of comprehensively deduping
equivalent TSs during post-hoc reads perhaps we wouldnt need this.
bq. On 2010-07-22 06:57:10, stack wrote:
bq. > src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java, line
405
bq. > <http://review.hbase.org/r/354/diff/1/?file=3018#file3018line405>
bq. >
bq. > Nice test
thanks, the EnvironmentEdge stuff is paying off already.
- Ryan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/354/#review450
-----------------------------------------------------------
> HBASE-2553 removed an important edge case
> ------------------------------------------
>
> Key: HBASE-2863
> URL: https://issues.apache.org/jira/browse/HBASE-2863
> Project: HBase
> Issue Type: Bug
> Reporter: ryan rawson
> Assignee: ryan rawson
> Fix For: 0.90.0
>
>
> in HBASE-2553 an important edge case whereby a KV with the same TS in
> snapshot was lost, tests have been failing (but flakly so) indicating it as
> well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
