also - how long are your column family name and column qualifiers? they are added to each row key in the index, so you want to make them as short as possible
On Wed, May 25, 2011 at 10:47 AM, Matt Corgan <[email protected]> wrote: > I have a table that compresses by 30x using gzip, so the default block size > of 64 KB was writing 2 KB blocks to disk. To reduce storefileIndexSize, I > raised the block size to 256 KB, presumably writing ~8KB disk blocks which > is still pretty small. Maybe you could go even higher depending on your > compression ratio. > > btw - why 10394 regions with only 3103 storefiles? > > > > 2011/5/25 Gaojinchao <[email protected]> > >> Region size is 512M >> >> hbase.regionserver.handler.count 50 >> hbase.regionserver.global.memstore.upperLimit 0.4 >> hbase.regionserver.global.memstore.lowerLimit 0.35 >> hbase.hregion.memstore.flush.size 128M >> hbase.hregion.max.filesize 512M >> hbase.client.scanner.caching 1 hfile.block.cache.size 0.2 >> hbase.hregion.memstore.block.multiplier 3 >> hbase.hstore.blockingStoreFiles 10 >> hbase.hstore.compaction.min.size 64M >> >> compress: gz >> >> dfs.block.size 256M >> >> -----邮件原件----- >> 发件人: [email protected] [mailto:[email protected]] 代表 Stack >> 发送时间: 2011年5月25日 11:57 >> 收件人: [email protected] >> 主题: Re: a question storefileIndexSize >> >> 2011/5/24 Gaojinchao <[email protected]>: >> > Stack, Thanks for your reply. >> > block size is default. >> > My Key length is 26 bytes and value is 300~400 bytes. >> > Is it big keys and small values ? >> > >> >> Looks like you have 'small' keys. >> >> It looks like the index is about 1MB per storefile (storefiles=3103, >> storefileIndexSize=3717). Does this seem about right? What size are >> your regions? >> >> St.Ack >> > >
