By using the block cache, read blocks are referenced within the block cache data structures and referenced for a longer amount of time than if not put into the block cache.
This will definitely add additional stress to the GC. If you expect a very low hit ratio, it can be advantageous to not use the block cache. You can also turn the block cache off on a per-query basis with setCacheBlocks(), though that's only supported on Scan right now. Still makes sense for Gets so we should add it there too. JG > -----Original Message----- > From: ChingShen [mailto:[email protected]] > Sent: Thursday, October 21, 2010 4:27 AM > To: [email protected] > Subject: Re: The hfile.block.cache.size = 0 performance is better than > default(0.2) in random read? Is it possible? > > Hi Ryan, > > *hfile.block.cache.size = 0, GC log:* > 2010-10-21T15:53:27.486+0800: 1428.317: [GC [PSYoungGen: > 18270K->320K(17728K)] 62008K->44066K(61696K), *0.0043520 secs*] [Times: > user=0.00 sys=0.00, real=0.00 secs] > 2010-10-21T15:53:27.933+0800: 1428.764: [GC [PSYoungGen: > 17641K->256K(17024K)] 61386K->44025K(60992K), *0.0036030 secs*] [Times: > user=0.00 sys=0.00, real=0.00 secs] > 2010-10-21T15:53:28.380+0800: 1429.212: [GC [PSYoungGen: > 17024K->288K(19648K)] 60793K->44130K(63616K), *0.0044410 secs*] [Times: > user=0.00 sys=0.00, real=0.01 secs] > 2010-10-21T15:53:28.385+0800: 1429.216: [Full GC [PSYoungGen: > 288K->0K(19648K)] [PSOldGen: 43841K->32920K(41536K)] 44130K- > >32920K(61184K) > [PSPermGen: 15684K->15683K(24640K)], *0.0480800 secs*] [Times: > user=0.05 > sys=0.00, real=0.05 secs] > > *hfile.block.cache.size = 0.2, GC log:* > 2010-10-21T16:18:31.884+0800: 1234.166: [GC [PSYoungGen: > 469577K->182750K(534208K)] 1183254K->1013795K(1663424K), *0.1265180 > secs*] > [Times: user=0.49 sys=0.00, real=0.13 secs] > 2010-10-21T16:18:56.837+0800: 1259.119: [GC [PSYoungGen: > 460382K->179115K(451392K)] 1291427K->1116923K(1580608K), *0.1231190 > secs*] > [Times: user=0.48 sys=0.00, real=0.13 secs] > 2010-10-21T16:19:20.121+0800: 1282.403: [GC [PSYoungGen: > 451371K->175649K(510016K)] 1389179K->1206321K(1639232K), *0.1153410 > secs*] > [Times: user=0.31 sys=0.01, real=0.11 secs] > 2010-10-21T16:19:20.236+0800: 1282.518: [Full GC [PSYoungGen: > 175649K->0K(510016K)] [PSOldGen: 1030672K->582437K(1179200K)] > 1206321K->582437K(1689216K) [PSPermGen: 16041K->16041K(21248K)], > *0.2538730 > secs*] [Times: user=0.26 sys=0.00, real=0.26 secs] > > hfile.block.cache.size = 0: > avg. per minor gc ~ *4ms* > avg. per full gc ~ *50ms* > > hfile.block.cache.size = 0.2: > avg. per minor gc ~* 120ms* > avg. per full gc ~ *250ms * > > Does it mean that because I got a low hit ratio in random read, so the > LruBlockCache object creates too many CachedBlock objects? > > Thanks. > > Shen > > On Thu, Oct 21, 2010 at 2:20 PM, Ryan Rawson <[email protected]> > wrote: > > > block cache invalidation is done async and isnt done inline, so that > > shouldn't be an issue. > > > > could be related to GC... if you could produce and compare GC logs > > that'd be helpful. > > > >
