Good point Matt. I forgot about compression. Let me add not to the above referenced section in the book.... St.Ack
On Wed, May 25, 2011 at 7:47 AM, Matt Corgan <[email protected]> wrote: > I have a table that compresses by 30x using gzip, so the default block size > of 64 KB was writing 2 KB blocks to disk. To reduce storefileIndexSize, I > raised the block size to 256 KB, presumably writing ~8KB disk blocks which > is still pretty small. Maybe you could go even higher depending on your > compression ratio. > > btw - why 10394 regions with only 3103 storefiles? > > > 2011/5/25 Gaojinchao <[email protected]> > >> Region size is 512M >> >> hbase.regionserver.handler.count 50 >> hbase.regionserver.global.memstore.upperLimit 0.4 >> hbase.regionserver.global.memstore.lowerLimit 0.35 >> hbase.hregion.memstore.flush.size 128M >> hbase.hregion.max.filesize 512M >> hbase.client.scanner.caching 1 hfile.block.cache.size 0.2 >> hbase.hregion.memstore.block.multiplier 3 >> hbase.hstore.blockingStoreFiles 10 >> hbase.hstore.compaction.min.size 64M >> >> compress: gz >> >> dfs.block.size 256M >> >> -----邮件原件----- >> 发件人: [email protected] [mailto:[email protected]] 代表 Stack >> 发送时间: 2011年5月25日 11:57 >> 收件人: [email protected] >> 主题: Re: a question storefileIndexSize >> >> 2011/5/24 Gaojinchao <[email protected]>: >> > Stack, Thanks for your reply. >> > block size is default. >> > My Key length is 26 bytes and value is 300~400 bytes. >> > Is it big keys and small values ? >> > >> >> Looks like you have 'small' keys. >> >> It looks like the index is about 1MB per storefile (storefiles=3103, >> storefileIndexSize=3717). Does this seem about right? What size are >> your regions? >> >> St.Ack >> >
