Thanks for all the feedback. We're going to spin up 0.20 to see if the scanner time improves. I'll try to get the times for scanner start-up (as a percentage of total time) when we run those tests.
Much appreciated! On Mon, Aug 3, 2009 at 3:34 PM, stack<[email protected]> wrote: > On Mon, Aug 3, 2009 at 1:58 PM, Xinan Wu <[email protected]> wrote: > >> 6 sec isn't crazy with 0.19. If you really want to research it, have a >> look at where the time is spent, creating scanner or actually doing >> the scanning. I think it's the former. > > > You are probably right that it is the fomer. In 0.19, scanners would open a > new Reader against every file in the region before scanning could start > (trip to namenode, then out to each dn to read in indices, etc.). In > 0.20.0, the already-open files are used. > St.Ack >
