junegunn commented on PR #8485:
URL: https://github.com/apache/hbase/pull/8485#issuecomment-5200797814
I extended `testSeeksEagerlyWhenFiltered` to sweep the row count, and turned
on the class's existing `VERBOSE` flag so it reports the seek count for the
whole scan rather than only at scanner open.
```diff
@@ -77,10 +77,11 @@ public class TestSeekOptimizations {
private static final int PUTS_PER_ROW_COL = 50;
private static final int DELETES_PER_ROW_COL = 10;
- private static final int NUM_ROWS = 3;
+ // Single digit only: rowStr() is "row" + i, so row10 would sort before
row2.
+ private static final int NUM_ROWS = 9;
private static final int NUM_COLS = 3;
- private static final boolean VERBOSE = false;
+ private static final boolean VERBOSE = true;
/**
* Disable this when this test fails hopelessly and you need to debug a
simpler case.
@@ -235,7 +236,7 @@ public class TestSeekOptimizations {
columnArr.length == 0 ? "all columns" : ("columns=" +
Arrays.toString(columnArr));
final String testDesc = "Bloom=" + bloomType + ", compr=" + comprAlgo +
", "
+ (scan.isGetScan() ? "Get" : "Scan") + ": " + columnRestrictionStr +
", " + rowRestrictionStr
- + ", maxVersions=" + maxVersions + ", lazySeek=" + lazySeekEnabled;
+ + ", maxVersions=" + maxVersions + ", lazySeek=" + lazySeekEnabled +
", filtered=" + filtered;
long seekCount = StoreFileScanner.getSeekCount() - initialSeekCount;
if (VERBOSE) {
System.err.println("Seek count: " + seekCount + ", KVs returned: " +
actualKVs.size() + ". "
@@ -467,6 +468,12 @@ public class TestSeekOptimizations {
public void testSeeksEagerlyWhenFiltered() throws IOException {
ScanResult filteredLazyResults = testScan(new int[] { 0 }, true, 0, 2,
1, true);
ScanResult filteredEagerResults = testScan(new int[] { 0 }, false, 0,
2, 1, true);
+ // Does the extra cost scale with rows scanned? filtered=false is the
behavior before
+ // this patch. endRow never equals startRow: that would be a Get, which
this patch exempts.
+ for (int endRow : new int[] { 1, 2, 5, 8 }) {
+ testScan(new int[] { 0 }, true, 0, endRow, 1, false);
+ testScan(new int[] { 0 }, true, 0, endRow, 1, true);
+ }
assertKVListsEqual("Filtered explicit column scan results differ with
lazy seeking enabled",
filteredEagerResults.cells, filteredLazyResults.cells);
assertEquals(filteredEagerResults.scannerOpenSeekCount,
```
Distilling the `columns=[0]` lines out of the output, identical for every
bloom type and codec:
| rows | lazy (`filtered=false`) | eager (`filtered=true`) | delta |
|---|---|---|---|
| 2 | 15 | 35 | 20 |
| 3 | 22 | 49 | 27 |
| 6 | 43 | 91 | 48 |
| 9 | 64 | 133 | 69 |
The fixture has 7 store files, and the numbers fit exactly:
```
lazy = 7*rows + 1
eager = 14*rows + 7
```
So in this fixture the eager initial seek does not cost a fixed N seeks at
open. It doubles the per-row seek count for the whole scan. That contradicts
the description:
> The cost is proportional to the number of underlying scanners opened, not
the number of rows returned.
and it also inverts what I said earlier about long scans amortizing the cost
away.
I haven't worked out what's causing the doubling, could you take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
