[
https://issues.apache.org/jira/browse/HBASE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128059#comment-13128059
]
[email protected] commented on HBASE-2856:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2224/
-----------------------------------------------------------
(Updated 2011-10-15 04:08:41.977544)
Review request for Ted Yu, Michael Stack, Kannan Muthukkaruppan, and Karthik
Ranganathan.
Changes
-------
Fix 2 more issues that could potentially cause ACID violations
(a) We only used to write maxVersions number of KV's to disk during
flush.
Not all KVs should be counted during this calculation. We shall
ignore all KV's newer than the oldest read point. So the oldest
Scanner can also get enough versions.
(b) move the ignoring newer KV's logic to the StoreFileScanner. That
way, this only returns KV's that are guaranteed to be included in the
search.
There was a condition where if two KVs were written to the same file. Both
identical, but only differ in memstoreTS, then we would skip the duplicate.
It was possible that the first one would be ignored because it has a newer
memstoreTS, and we would never get to see the second one, which might be
the KV we want.
Summary
-------
address the 2856 issues by writing the memstoreTS to the disk.
version v11 of the patch.
uploading it here for easier review process.
This addresses bug HBASE-2856.
https://issues.apache.org/jira/browse/HBASE-2856
Diffs (updated)
-----
/pom.xml 1183581
/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java
1183581
/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 1183581
/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java 1183581
/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java 1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnCount.java 1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java 1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/ReadWriteConsistencyControl.java
1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
1183581
/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java 1183581
/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 1183581
/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java 1183581
/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java 1183581
/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java
1183581
/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
1183581
/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java
1183581
/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
1183581
/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
1183581
Diff: https://reviews.apache.org/r/2224/diff
Testing
-------
mvn test
Thanks,
Amitanand
> TestAcidGuarantee broken on trunk
> ----------------------------------
>
> Key: HBASE-2856
> URL: https://issues.apache.org/jira/browse/HBASE-2856
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.89.20100621
> Reporter: ryan rawson
> Assignee: Amitanand Aiyer
> Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: 2856-v2.txt, 2856-v3.txt, 2856-v4.txt, 2856-v5.txt,
> acid.txt
>
>
> TestAcidGuarantee has a test whereby it attempts to read a number of columns
> from a row, and every so often the first column of N is different, when it
> should be the same. This is a bug deep inside the scanner whereby the first
> peek() of a row is done at time T then the rest of the read is done at T+1
> after a flush, thus the memstoreTS data is lost, and previously 'uncommitted'
> data becomes committed and flushed to disk.
> One possible solution is to introduce the memstoreTS (or similarly equivalent
> value) to the HFile thus allowing us to preserve read consistency past
> flushes. Another solution involves fixing the scanners so that peek() is not
> destructive (and thus might return different things at different times alas).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
