I tried 1024 & 2048 bytes value size , but the space amplification is ~2, so If I want to reduce this space amplification, what's your suggestion?
Cheers, xinxin -----Original Message----- From: Howard Chu [mailto:[email protected]] Sent: Wednesday, April 22, 2015 3:48 AM To: Shu, Xinxin; [email protected] Subject: Re: LMDB space amplification Shu, Xinxin wrote: > Hi list, > > Recently I have integrated lmdb into ceph , got intial results , I > dump all records of lmdb, there are several different size key-value > pairs, > > 1. key size : 45 byte, value size = 124 byte 2. key size : 85 byte, > value size = 1187 byte 3. key size : 57 byte, value size = 135 byte > 4. key size : 64 byte, value size = 182 byte 5. key size : 33 byte, > value size = 4096 byte (used for real data, about 80 %) > > this db contains about ~ 10000000 records, about 80 percent of records are > the last key-value pairs (key size 33 byte, value size 4096 byte), I > calculate size of all key-value pairs, about 33GB , but on-disk size is > about 63GB, so space amplification is ~ 2, is this amplification reasonable ? > In which situation can lmdb get such large space amplification? How can I > reduce this amplification? If you need any other details , please email me , > any tips will be appreciate. LMDB pages have a 16 byte header. In a system with 4096 byte pages, only 4080 bytes are available for user data, so storing a data value of 4096 bytes will use 2 pages. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
