[ https://issues.apache.org/jira/browse/HBASE-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837519#action_12837519 ]
Dave Latham commented on HBASE-2248: ------------------------------------ @Dan: Took a read over the patch, though it seemed to be based in a different dir and didn't want to apply nicely. From what I can see the ScanTest still does getScanner in testSetup before the timer is begun. This may be fine, if the point of this test is to measure scan performance per-row and not setup/teardown time. It just explains why the ScanTest doesn't exhibit this issue. It does look like other tests, such as the RandomSeekScanTest and the new RandomScanWithRangeTest do test setup/teardown time as part of each "testRow" and should exhibit this issue, if run. > New MemStoreScanner copies memstore for each scan, makes short scans slow > ------------------------------------------------------------------------- > > Key: HBASE-2248 > URL: https://issues.apache.org/jira/browse/HBASE-2248 > Project: Hadoop HBase > Issue Type: Bug > Affects Versions: 0.20.3 > Reporter: Dave Latham > Fix For: 0.20.4 > > Attachments: hbase-2248.gc, Screen shot 2010-02-23 at 10.33.38 > AM.png, threads.txt > > > HBASE-2037 introduced a new MemStoreScanner which triggers a > ConcurrentSkipListMap.buildFromSorted clone of the memstore and snapshot when > starting a scan. > After upgrading to 0.20.3, we noticed a big slowdown in our use of short > scans. Some of our data repesent a time series. The data is stored in time > series order, MR jobs often insert/update new data at the end of the series, > and queries usually have to pick up some or all of the series. These are > often scans of 0-100 rows at a time. To load one page, we'll observe about > 20 such scans being triggered concurrently, and they take 2 seconds to > complete. Doing a thread dump of a region server shows many threads in > ConcurrentSkipListMap.biuldFromSorted which traverses the entire map of key > values to copy it. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.