[
https://issues.apache.org/jira/browse/HBASE-5010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HBASE-5010:
-------------------------------
Attachment: D909.1.patch
mbautin requested code review of "[jira] [HBASE-5010] [89-fb] Filter HFiles
based on TTL".
Reviewers: Kannan, Liyin, JIRA
Modifying scanner selection in StoreScanner to take TTL into account, so that
we don't scan StoreFiles that only contain expired keys.
This diff is for 89-fb, but there will be a very similar diff for trunk.
TEST PLAN
Unit tests (existing ones and a new one).
Deploy to cluster and scan an existing table that only contains expired keys.
Verify that next() calls don't time out. Previously, the scanner would try to
go through the whole table in an attempt to find a non-expired key.
REVISION DETAIL
https://reviews.facebook.net/D909
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueScanner.java
src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
src/main/java/org/apache/hadoop/hbase/regionserver/NonLazyKeyValueScanner.java
src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
src/main/java/org/apache/hadoop/hbase/util/Threads.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestSelectScannersUsingTTL.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1911/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Filter HFiles based on TTL
> --------------------------
>
> Key: HBASE-5010
> URL: https://issues.apache.org/jira/browse/HBASE-5010
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Attachments: D909.1.patch
>
>
> In ScanWildcardColumnTracker we have
> {code:java}
>
> this.oldestStamp = EnvironmentEdgeManager.currentTimeMillis() - ttl;
> ...
> private boolean isExpired(long timestamp) {
> return timestamp < oldestStamp;
> }
> {code}
> but this time range filtering does not participate in HFile selection. In one
> real case this caused next() calls to time out because all KVs in a table got
> expired, but next() had to iterate over the whole table to find that out. We
> should be able to filter out those HFiles right away. I think a reasonable
> approach is to add a "default timerange filter" to every scan for a CF with a
> finite TTL and utilize existing filtering in
> StoreFile.Reader.passesTimerangeFilter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
