We have been doing some performance testing on HBase filters. One outcome was HBASE-4626 (which I fixed and committed yesterday night).
Now we found a rather strange behavior with SingleColumnValueFilter. On our test cluster it is 10x slower than ValueFilter, even when we restrict the scan to just the one column we are filtering on and set filterIfMissing to true. We are not seeing that with HBase in local mode, which points to some additional activity on the FS, which in HDFS would be slow compared to a local FS. Indeed it turns out the problem goes away when we replace all NEXT_ROW with SKIP in SingleColumnValueFilter.filterKeyValue the performance is *much* better (on par with ValueFilter). We're using something pretty close to trunk for our tests. The tables are pretty wide, only one version of each cells (and freshly major compacted). I do not know this part of the code that well (yet) and was wondering if somebody could chime in. Maybe this is related to HFileV2? I do recall there was something done to optimize reseeks. Generally I would have expected NEXT_ROW to be a major performance improvement. Any ideas, comments, pointers? Thanks. -- Lars
