Here's a curious case that I had not encountered with LMDB as yet previously:
1. There was a power reset of a virtual machine with an active LMDB writer process (standalone use, not OpenLDAP) on an LMDB file containing three sub-DBs. 2. After rebooting, the previously-populated LMDB file (~7 GB in size) appears mostly empty, including when examined with mdb_stat or mdb_dump. Mostly empty meaning that each of three sub-DBs now has only one K/V entry, instead of 7M+ as they used to. In addition, the main DB now indicates six entries instead of the expected three (for the sub-DBs). 3. mdb_copy (with or without -c) does not remedy the situation, producing a mostly (logically) empty database. This is with LMDB release 0.9.18 running on Ubuntu 14.04.4 (kernel 3.13.0-79-generic) on an ext4 partition (noatime,nodev,nosuid,noexec) on Intel SSD storage in SW RAID-1 configuration. As mentioned, the LMDB file had three sub-DBs, each with 7M+ entries (as of last backup). No new sub-DBs are created after the database is initially initialized. After initial creation, these three sub-DBs only ever get appended to with new key/value pairs, no code ever deletes or modifies key/value pairs in them. The writer code inserts new entries one at a time, commits the LMDB transaction, and syncs to disk. I've enclosed mdb_stat output from before/after (before being from a backup, after which numerous more writes had been done). I've also included mdb_dump output of the main DB and three sub-DBs. The mdb_dump output for the sub-DBs indicates that they each now contain only a single entry (instead of 7M+), that entry being in each case the first key/value pair that was ever inserted into that sub-DB (ages ago). The mdb_dump output for the main DB is baffling--instead of the three expected entries, or the six that mdb_stat indicates after the reboot, the output includes a multitude of entries--some 2,590. (I've omitted most of them in the attached, but can provide a copy privately.) What are my options for recovering an LMDB database in this state, to the extent possible? Has anyone else experienced a similar scenario? Thanks, Arto -- Arto Bendiken | @bendiken | http://ar.to
### BEFORE POWER OUTAGE (from last backup) $ mdb_stat -ea . Environment Info Map address: (nil) Map size: 1099511627776 Page size: 4096 Max pages: 268435456 Number of pages used: 272654 Last transaction ID: 2210167 Max readers: 4096 Number of readers used: 0 Status of Main DB Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 3 Status of sha1:u32 Tree depth: 4 Branch pages: 1043 Leaf pages: 88442 Overflow pages: 0 Entries: 7324562 Status of u32:cstr Tree depth: 4 Branch pages: 419 Leaf pages: 120625 Overflow pages: 344 Entries: 7324562 Status of u32:sha1 Tree depth: 3 Branch pages: 214 Leaf pages: 61551 Overflow pages: 0 Entries: 7324562 ### AFTER POWER OUTAGE $ ls -l *.mdb -rw-rw-r-- 1 root root 7448719360 Mar 14 15:25 data.mdb -rw-rw-r-- 1 root root 262272 Mar 14 16:13 lock.mdb $ mdb_stat -ea . Environment Info Map address: (nil) Map size: 1099511627776 Page size: 4096 Max pages: 268435456 Number of pages used: 1818535 Last transaction ID: 4132159 Max readers: 4096 Number of readers used: 0 Status of Main DB Tree depth: 1 Branch pages: 0 Leaf pages: 2 Overflow pages: 0 Entries: 6 Status of sha1:u32 Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 1 Status of u32:cstr Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 1 Status of u32:sha1 Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 1 $ mdb_dump -s sha1:u32 . VERSION=3 format=bytevalue database=sha1:u32 type=btree mapsize=1099511627776 maxreaders=4096 db_pagesize=4096 HEADER=END da39a3ee5e6b4b0d3255bfef95601890afd80709 00000000 DATA=END $ mdb_dump -s u32:cstr . VERSION=3 format=bytevalue database=u32:cstr type=btree mapsize=1099511627776 maxreaders=4096 integerkey=1 db_pagesize=4096 HEADER=END 00000000 00 DATA=END $ mdb_dump -s u32:sha1 . VERSION=3 format=bytevalue database=u32:sha1 type=btree mapsize=1099511627776 maxreaders=4096 integerkey=1 db_pagesize=4096 HEADER=END 00000000 da39a3ee5e6b4b0d3255bfef95601890afd80709 DATA=END $ mdb_dump . VERSION=3 format=bytevalue type=btree mapsize=1099511627776 maxreaders=4096 db_pagesize=4096 HEADER=END 54c93c0000000000 0a0000000000000071660a000000000069660a000000000066660a000000000054660a00000000000f650a0000000000d4630a00000000008b610a0000000000875f0a0000000000cf4d0a0000000000a2ac090000000000 55c93c0000000000 0b00000000000000770f1800000000000d5a17000000000080660a00000000007c660a000000000075660a000000000074660a000000000072660a00000000006d660a00000000006b660a00000000004e3b0a00000000008f0e0a0000000000 56c93c0000000000 ... 2585 entries omitted ... 0a00000000000000c2d00a0000000000c1d00a0000000000bad00a0000000000b0d00a0000000000afd00a00000000005f5b0a0000000000c9410a00000000006c6a09000000000004590900000000000ec9070000000000 6dd33c0000000000 0a00000000000000c3d00a0000000000c0d00a0000000000bfd00a0000000000b9d00a0000000000b8d00a000000000091d00a000000000090d00a000000000082d00a00000000007cd00a00000000000ed9030000000000 6ed33c0000000000 0900000000000000c7d00a0000000000c4d00a00000000009fd00a000000000099d00a000000000098d00a000000000081d00a000000000064cf0a0000000000519a090000000000b366000000000000 6fd33c0000000000 0a0000000000000042641700000000005874110000000000e0d00a0000000000c9d00a0000000000c5d00a000000000049cf0a0000000000299e0a0000000000916a0a0000000000ae360a0000000000a8130a0000000000 DATA=END
