> On 2010-10-31 13:32:44, khemani wrote: > > The timestamp that we put in the column-qualifier to create hourly counters > > need not be in sync with the KV timestamp. This is because there are times > > when the log stream falls behind and we might be updating couple of hours > > old counters. The time-range that we provide has to be dynamically > > determined based on the current log-stream delay. > > > > > > This will really work well if along with hourly counters we also have > > hourly store files. If everything gets compacted into a single store file > > then this change doesn't help much. > >
Yeah, if you aren't doing all of your increments at the same time as the stamps they represent, you'll need to modify the TimeRange. Something like: [min,max) -> [minStampInPartition,Long.MAX_VALUE) where minStampInPartition is the lowest timestamp possible for the time bucket you are incrementing. As we begin to grow a large amount of historical data, it will be important that our compaction policy eventually just archives old data and it does not get included in further compactions. This TimeRange functionality will ensure they don't impact performance on new data. - Jonathan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1132/#review1731 ----------------------------------------------------------- On 2010-10-29 19:42:21, Jonathan Gray wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > http://review.cloudera.org/r/1132/ > ----------------------------------------------------------- > > (Updated 2010-10-29 19:42:21) > > > Review request for hbase, stack, Ryan Rawson, and khemani. > > > Summary > ------- > > TimeRange now part of Increment. Adds new methods into Increment w/ javadoc. > Adds code in our increment operation to set this onto the Gets we use. > > > This addresses bug HBASE-3162. > http://issues.apache.org/jira/browse/HBASE-3162 > > > Diffs > ----- > > trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java 1028936 > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > 1028936 > > Diff: http://review.cloudera.org/r/1132/diff > > > Testing > ------- > > increment tests passing. > > > Thanks, > > Jonathan > >
