Re: I killed my RPMDB --- any rescue possible?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sunday 12 June 2011, Jeff Johnson n3...@mac.com wrote: However, after using rpm -r /my/chroot -Uvh ..., the RPMDB *outside* the chroot is corrupted. When running rpm -qa, this is what I get: So which rpmdb needs fixing? The rpm inside or outside the chroot? Outside. The host system's RPMDB is broken (after rpm -r ... -Uvh ...). The chroot's RPMDB is intact, but the package I wanted to install is missing, not so surprisingly. :-) And what versions of rpm are in use inside and outside the chroot? 5.3.10 in both cases. Underlying what? Sorry, wrong wording: The bdb version linked with the current rpm-5.3.10 install is 5.1.25. The RPMDB was originally created using bdb 4.7.something, and after installing rpm-5.3.10 (with bdb-5.1.25), I immediateley ran dbconvert.sh. Clean-up and save what you have in /var/lib/rpm (...) Create a new directory and copy what you have (...) Check (as root) that Packages is at least mostly intact: rpm -qavv I did as advised, and rpm -qavv outputs this: - ---%--- D: pool fd: created size 208 limit -1 flags 0 D: pool iob:created size 20 limit -1 flags 0 D: pool mire: created size 84 limit -1 flags 0 D: pool lua:created size 32 limit -1 flags 0 D: pool ts: created size 720 limit -1 flags 0 D: pool gi: created size 92 limit -1 flags 0 D: pool db: created size 184 limit -1 flags 0 D: pool dbi:created size 284 limit -1 flags 0 D: opening db index /var/lib/rpm/Packages thread:rdonly mode=0x0 D: pool mi: created size 88 limit -1 flags 0 D: pool h: created size 216 limit -1 flags 0 libXfixes-4.0.3-1ev.i686 error: rpmdb: header #33554432 cannot be loaded -- skipping. (none)-(none)-(none) == munmap(0xb65d[180811702]) error(22): Invalid argument error: rpmdb: header #67108864 cannot be loaded -- skipping. error: rpmdb: header #83886080 cannot be loaded -- skipping. error: rpmdb: header #100663296 cannot be loaded -- skipping. error: rpmdb: header #117440512 cannot be loaded -- skipping. error: rpmdb: header #134217728 cannot be loaded -- skipping. error: rpmdb: header #150994944 cannot be loaded -- skipping. error: rpmdb: header #167772160 cannot be loaded -- skipping. error: rpmdb: header #201326592 cannot be loaded -- skipping. error: rpmdb: header #218103808 cannot be loaded -- skipping. error: rpmdb: header #234881024 cannot be loaded -- skipping. error: rpmdb: header #268435456 cannot be loaded -- skipping. error: rpmdb: header #285212672 cannot be loaded -- skipping. error: rpmdb: header #301989888 cannot be loaded -- skipping. (none)-(none)-(none) (none)-(none)-(none) (none)-(none)-(none) error: rpmdb: header #385875968 cannot be loaded -- skipping. (none)-(none)-(none) Segmentation fault - ---%--- Re-create the indices (and check that dbconvert.sh swapped the bytes): rpm --rebuilddb --vv - ---%--- D: pool fd: created size 208 limit -1 flags 0 D: pool iob:created size 20 limit -1 flags 0 D: pool mire: created size 84 limit -1 flags 0 D: pool lua:created size 32 limit -1 flags 0 D: pool ts: created size 720 limit -1 flags 0 D: pool db: created size 184 limit -1 flags 0 D: pool dbi:created size 284 limit -1 flags 0 D: rpmdb: cpus 2 physmem 3026Mb D: opening db environment /var/lib/rpm/Packages thread:lock:log:mpool:txn rpmdb: Thread/process 18824/3062732608 failed: Thread died in Berkeley DB library error: db_init:db3.c:1020: dbenv-failchk(-30973): DB_RUNRECOVERY: Fatal error, run database recovery Re-opening dbenv with DB_RECOVER ... D: opening db environment /var/lib/rpm/Packages thread:lock:log:mpool:txn:recover Finding last valid log LSN: file: 1 offset 9104 recovery 28% completeRecovery starting from [1][162] recovery 95% completeRecovery complete at Mon Jun 13 13:54:23 2011 Maximum transaction ID 8003 Recovery checkpoint [1][9296] . recovery succeeded. D: opening db index /var/lib/rpm/Packages create:thread:auto_commit mode=0x2 D: opening db index /var/lib/rpm/Name create:thread:auto_commit mode=0x2 D: pool h: created size 216 limit -1 flags 0 error: db3: header #33554432 cannot be loaded -- skipping. error: db3: header #67108864 cannot be loaded -- skipping. error: db3: header #83886080 cannot be loaded -- skipping. error: db3: header #100663296 cannot be loaded -- skipping. error: db3: header #117440512 cannot be loaded -- skipping. error: db3: header #134217728 cannot be loaded -- skipping. error: db3: header #150994944 cannot be loaded -- skipping. error: db3: header #167772160 cannot be loaded -- skipping. error: db3: header #201326592 cannot be loaded -- skipping. error: db3: header #218103808 cannot be loaded -- skipping. error: db3: header #234881024 cannot be loaded -- skipping. error: db3: header #268435456 cannot be loaded -- skipping. error: db3: header #285212672 cannot be loaded -- skipping. error: db3: header #301989888 cannot be loaded -- skipping. error: db3: header
Re: I killed my RPMDB --- any rescue possible?
On Jun 13, 2011, at 9:58 AM, Eric MSP Veith wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sunday 12 June 2011, Jeff Johnson n3...@mac.com wrote: However, after using rpm -r /my/chroot -Uvh ..., the RPMDB *outside* the chroot is corrupted. When running rpm -qa, this is what I get: So which rpmdb needs fixing? The rpm inside or outside the chroot? Outside. The host system's RPMDB is broken (after rpm -r ... -Uvh ...). The chroot's RPMDB is intact, but the package I wanted to install is missing, not so surprisingly. :-) OK. And what versions of rpm are in use inside and outside the chroot? 5.3.10 in both cases. OK. Underlying what? Sorry, wrong wording: The bdb version linked with the current rpm-5.3.10 install is 5.1.25. The RPMDB was originally created using bdb 4.7.something, and after installing rpm-5.3.10 (with bdb-5.1.25), I immediateley ran dbconvert.sh. OK. There are no significant format changes in Berkeley DB that are NOT handled by db-utils, specifically db_dump … | db_load. What is subtle is that one MUST choose the correct version specific db_dump/db_load to upgrade (and downgrade). I'm not at all sure that dbconvert.sh is properly implemented to Just Work with 1) a outer - inner chroot system 2) multiple rpmdb paths 3) multiple database versions The safest way to avoid problems is to do chroot /your/chroot /bin/bash and just run dbconvert.sh within. Clean-up and save what you have in /var/lib/rpm (...) Create a new directory and copy what you have (...) Check (as root) that Packages is at least mostly intact: rpm -qavv I did as advised, and rpm -qavv outputs this: - ---%--- D: pool fd: created size 208 limit -1 flags 0 D: pool iob:created size 20 limit -1 flags 0 D: pool mire: created size 84 limit -1 flags 0 D: pool lua:created size 32 limit -1 flags 0 D: pool ts: created size 720 limit -1 flags 0 D: pool gi: created size 92 limit -1 flags 0 D: pool db: created size 184 limit -1 flags 0 D: pool dbi:created size 284 limit -1 flags 0 D: opening db index /var/lib/rpm/Packages thread:rdonly mode=0x0 D: pool mi: created size 88 limit -1 flags 0 D: pool h: created size 216 limit -1 flags 0 libXfixes-4.0.3-1ev.i686 error: rpmdb: header #33554432 cannot be loaded -- skipping. (none)-(none)-(none) == munmap(0xb65d[180811702]) error(22): Invalid argument error: rpmdb: header #67108864 cannot be loaded -- skipping. error: rpmdb: header #83886080 cannot be loaded -- skipping. error: rpmdb: header #100663296 cannot be loaded -- skipping. error: rpmdb: header #117440512 cannot be loaded -- skipping. error: rpmdb: header #134217728 cannot be loaded -- skipping. error: rpmdb: header #150994944 cannot be loaded -- skipping. error: rpmdb: header #167772160 cannot be loaded -- skipping. error: rpmdb: header #201326592 cannot be loaded -- skipping. error: rpmdb: header #218103808 cannot be loaded -- skipping. error: rpmdb: header #234881024 cannot be loaded -- skipping. error: rpmdb: header #268435456 cannot be loaded -- skipping. error: rpmdb: header #285212672 cannot be loaded -- skipping. error: rpmdb: header #301989888 cannot be loaded -- skipping. (none)-(none)-(none) (none)-(none)-(none) (none)-(none)-(none) error: rpmdb: header #385875968 cannot be loaded -- skipping. (none)-(none)-(none) Segmentation fault - ---%--- OK. You have a damaged database for whatever reason. All of those header #12345678 are too big so you likely have NOT succeeded in byte swabbing the header instances (i.e. what dbconvert.sh is doing). (alternatively) You have run dbconvert.sh an even number of times and so two applications of dbconvert.sh has undone the byte swabbing originally intended. You can try reading the rpmdb with pre-rpm-5.3 to assess how much data is savable. Doing rpm --rebuilddb (possibly with --nosignature --nodigest --nohdrchk disablers to get past the segfault) WILL discard damaged headers that you can then replace later. While doing --rebuilddb, make sure you do either cd /var/lib/rpm rm -f __db* or cd /var/lib/rpm db_recover -v (note no -e passed to db_recover to remove the __db cache) A damaged cache in __db* files can/will screw a --rebuilddb worser. Re-create the indices (and check that dbconvert.sh swapped the bytes): rpm --rebuilddb --vv - ---%--- D: pool fd: created size 208 limit -1 flags 0 D: pool iob:created size 20 limit -1 flags 0 D: pool mire: created size 84 limit -1 flags 0 D: pool lua:created size 32 limit -1 flags 0 D: pool ts: created size 720 limit -1 flags 0 D: pool db: created size 184 limit -1 flags 0 D: pool dbi:created size 284 limit -1 flags 0 D: rpmdb: cpus 2 physmem 3026Mb D: opening db environment /var/lib/rpm/Packages thread:lock:log:mpool:txn rpmdb: Thread/process 18824/3062732608 failed: Thread
Re: I killed my RPMDB --- any rescue possible?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Monday 13 June 2011, Jeff Johnson n3...@mac.com wrote: (...) You can try reading the rpmdb with pre-rpm-5.3 to assess how much data is savable. (...) Ok, I salvaged what I could and replaced the rest from a backup. I found out that there (a) was a corrupted RPMDB, but re-running (b) dbconvert.sh later on in a futile attempt to fix it made things worse. So byteswapping and rebuilding largely helped. TA! Do you have any hint/idea why rpm -r /my/chroot -Uvh /some/package.rpm kills the RPMDB *outside* of the chroot? Did I miss anything here? I don't know what operations were run, and with what utilities, so I cannot hazard a guess. Only one, which is rpm -r /srv/vm/buildroot -Uvh $( ~/Packaging/baseset.txt). I.e., basic chroot populating. If you believe that rpm is confused about outer - inner chroot(2) behavior, then you need to run strace -e open -o /tmp/xyzzy -f rpm … and examine the paths that were opened. In _ALL_ cases, doing chroot /your/chroot /bin/bash and avoiding -r works. Ok, I'll do that. What's the preferred way to initially populate a chroot? TIA, Eric -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) iQIcBAEBAgAGBQJN9idTAAoJEIXD2TZ1epVB9KoP+wVB4YNEdGyMaA7FyLNG3Bc5 ZC5CUXf4nV8KD+YcBFfTgFdroGGF8vx8ZmcEl5zPQTloIorgpGEbwcd0AqtdCONl /nVsn1Q84o4F1O5Yxy+15IPaPJZKYrFBTdQvEriH9IepEKjwooDGW1MUID3SuPEq mbaLBJV6sym1cSBAkQ/bp/GO2BjSwAssuRiBjZX/k80d9p0Fa3E3tVvFRvq3YbaN pK7way2w7kgkl8eb/IlxTY9XdUYQQt/kfDVKRr7qVq/aascEypH/0/+a7OYJywFv VEz6O1kpC6WsKDwTB7zrR67rFkeMpiVxTWT0FwszGfnTZdfDNl5kcPmfeikZZkL5 idBKmIHTrkYyUgi4Uf2ea1+gLfZtUpARkapCUAh0WQ9Ca2ThG/uJViI3yJ6utI0Q /CFmDpTdjIUXwrNoEMR7vRSMYtqYfRY3i+h8U0hKSTgBRFS852RpSTY+acwi4zZ5 /R3zMJhOUtNmX04VZymZTl/KzfktQvZEBElBXdf9W5S2Nkqs8OrZzUqE41ymA8mI FPVETS8mETO/Y3uz8jtIBPAU3d7N97FTyxjTG5jL8Ll/gZTrjUFf0nPp7d/xxoj0 +U1HZQ2yDpSciecwddJEt4t5eO3AYFMkkRzZTO9ie7xGPpMt4G59xWv0tt5hxURC VBOn0dMT/BBbL84QC+7l =S9FW -END PGP SIGNATURE- __ RPM Package Managerhttp://rpm5.org User Communication List rpm-users@rpm5.org
Re: I killed my RPMDB --- any rescue possible?
On Jun 13, 2011, at 11:05 AM, Eric MSP Veith wrote: What's the preferred way to initially populate a chroot? The only difference in rpm-5.3 related ro your issue is running dbconvert.sh in %posttrans. The whole idea of converting an *active* rpmdb *automagically* is _INSANE_. There are far far too many error cases for any dbconvert.sh script to _GUARANTEE_ success _ALWAYS_. Everything is _EXACTLY_ as has always been implemented with rpm and chroot(2). There are no changes: A script/executable invoked while installing with rpm -r can escape a chroot and do whatever it wishes. 73 de Jeff smime.p7s Description: S/MIME cryptographic signature
Re: I killed my RPMDB --- any rescue possible?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Monday 13 June 2011, Jeff Johnson n3...@mac.com wrote: On Jun 13, 2011, at 11:05 AM, Eric MSP Veith wrote: What's the preferred way to initially populate a chroot? The only difference in rpm-5.3 related ro your issue is running dbconvert.sh in %posttrans. I wasn't refering to rpm5/dbconvert.sh in that paragraph of mine, but considering that I chose a wrong approach in the past with populating chroots _in general_. Sorry if I didn't express myself clearly. I do not trust dbconvert.sh to automagically find out whether a RPMDB needs converting or not, hence I do not call it from rpm's %post. What I wanted to ask was: Considering installing RPM packages into a system initially, such as it often happens when creating a build chroot, what's the best approach one can choose here? Everything is _EXACTLY_ as has always been implemented with rpm and chroot(2). There are no changes: A script/executable invoked while installing with rpm -r can escape a chroot and do whatever it wishes. Sure, in that case I gladly eat my hat or do something similar as to punish myself for my own stupidity since I'm the one creating %post co. :-) TIA, Eric -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) iQIcBAEBAgAGBQJN9izCAAoJEIXD2TZ1epVB32wP/R437nRb6P//wpz5FYncHGaU V+dSOw+enqnpZbi7JDUahzBLK9i4mSVWCC0I0fk8U1HgN7NzoBcYBQokg+aKmv5e GIm+6ngv5dAaT1ldVvENwxxR4HNRF9znnTs8OnRTLjF4rbwDowUGaPph8iuoY8Me TbW960eceygAPXhdpxCcu/imSkEBvubvqbXGt1pbpQZ+/iWucKq6N7KXCIFnWG+X i+UabOOHAq2wpGFFdxrK+Zmx4iMBYGcDLXj37m4LplW6OAZHE33ScjdJLAhF0VLr H/ZrCTKe6wctoHa6ccAacvYhXt7hahDKVoyDZNkVNFXV775rZMLqBv4JbYfH4bXZ XTxVGJjQ0PwoQmAsd8wnvo5daHji36eVRHUlGE75onnYlgIBiXG4KkEvlhewDymU LQD0TeP0bh5w6XMinoHe4+X+9gzEG4SqOlFm2XTQT3Hq5HHFkRRz5u95VT2LMmK+ pfZ4xa3TwiJ8bCQuou8p/cqP0fgAf8arJvAOeJN5unCG3lm5fXaqylgCFdaGIZKv QrB0aE7eGe0/IVm9YApRp0+YgiG7IVX71vOf3VeAM3aqsDboZznewusLUsYxconc DGN8Q6uL35A0V0SK/1c37WSnEBwF74nNR7/atfHYH6VjvfMblq8zckuUAypm5P6L WjkrAecFM3+P4cHEgVdl =Wa84 -END PGP SIGNATURE- __ RPM Package Managerhttp://rpm5.org User Communication List rpm-users@rpm5.org
