Wow that is brutal. Are you able to make a patch? How could we add a test to protect ourselves from these kinds of things?
On Tue, Feb 23, 2010 at 11:56 PM, Dan Washusen <d...@reactive.org> wrote: > It's an issue with WhileMatchFilter. I've created issue HBASE-2258... > > On 24 February 2010 18:25, Ryan Rawson <ryano...@gmail.com> wrote: > >> That is very interesting, there are definitely some hidden performance >> landmines in the filter code... you have clearly found one. >> >> strange in that while match filter is just a thin wrapper. >> >> >> >> On Tue, Feb 23, 2010 at 11:22 PM, Dan Washusen <d...@reactive.org> wrote: >> > OK, a bit more info; removing the WhileMatchFilter drops the scans down >> to >> > 25ms each. >> > >> > Updated scan looks like: >> > >> >> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows)); >> >> >> >> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME); >> >> >> >> scan.setFilter(new PageFilter(120)); >> >> >> >> >> > The slow scan uses: >> > >> >> Scan scan = new Scan(getRandomRow(this.rand, this.totalRows)); >> >> >> >> scan.addColumn(FAMILY_NAME, QUALIFIER_NAME); >> >> >> >> scan.setFilter(new *WhileMatchFilter*(new PageFilter(120))); >> >> >> >> >> > >> > >> > On 24 February 2010 18:17, Dan Washusen <d...@reactive.org> wrote: >> > >> >> Hey All, >> >> While testing the recent MemStoreScanner slowness I noticed that each >> scan >> >> in the randomSeekScan test takes about 19 seconds to complete. The scan >> in >> >> question provides a startRow and a WhileMatchFilter containing a >> PageFilter >> >> that asks for 120 rows. I would have expected this scan to return in >> >> roughly the same amount of time as a scan that specifies a startRow and >> >> stopRow that spans a similar number of rows. >> >> >> >> Before I start investigating further I was wondering if anyone had any >> >> thoughts? >> >> >> >> Cheers, >> >> Dan >> >> >> >> p.s. Running the test on the latest 0.20 code... >> >> >> > >> >
