Re: rpmdb segfaults after upgrade from rpm 4.6.0

2009-11-18 Thread Per Øyvind Karlsen
2009/11/18 Jeff Johnson n3...@mac.com

 So what is needed is a --rebuilddb with the PROT_WRITE added, and
 then revert to only setting PROT_READ. How you wish to achieve
 that conversion on the fly is up to you. The forward looking goal
 is to use PROT_READ hardware protection rather than all the
 overly complex and senseless digest/signature/sanity checking
 with is most definitely cpu intensive and largely unnecessary.

Hmm, if not automatically handling this in code, what about some option,
macro or something? It obviously seems like something that a lot of people
is likely to run into and therefore should be dealt with in a way easily
usable for everyone.


 So headerVerifyInfo() (and the attempt to change PROT_READ memory
 that results in a SIG11) is going into the bit bucket where
 signature/digest
 checking of rpmdb Headers has already been discarded.

 (aside)
 There's another way to fix this issue by re-adding RPM_CHAR_TYPE rather
 than attempting to change a data type in a PROT_READ header on the fly
 as is being done in rpmdb/header_internal.c.

 The goal in RPM-5.0 is/was to make all header data types unsigned,
 and that goal was most definitely achieved, and signified by doing a major
 release of RPM in which no compatibility was promised or intended.

 I'm not about to go all the way back and re-release every version of RPM
 since rpm-5.0
 to retrofit compatibility  for distros that don't use rpm5.orgcode 
 particularly since this issue
 was both known and discussed before rpm-5.0 was released. See rpm-devel
 archives.

 But a conversion tool could be written rather easily.

 The RPM_CHAR_TYPE is never found in package headers,
 is used solely in one place, adding RPMTAG_FILESTATES while
 installing by rpm-4.x.

 Here is the patch that fixes by re-adding RPM_CHAR_TYPE . There's
 additional places needed
 to remove compiler warnings (at a minimum), and likely to display
 RPM_CHAR_TYPE if
 the already removed data type is to be retrofitted.

Hmm, so if you'd like to attempt achieving some compatibility with rpm.org,
this would be preferred?


 $ cvs diff rpmtag.h header_internal.c
 Index: rpmtag.h
 ===
 RCS file: /v/rpm/cvs/rpm/rpmdb/rpmtag.h,v
 retrieving revision 1.62.2.2
 diff -p -u -w -r1.62.2.2 rpmtag.h
 --- rpmtag.h 25 Jul 2009 00:56:51 - 1.62.2.2
 +++ rpmtag.h 18 Nov 2009 03:58:11 -
 @@ -25,7 +25,7 @@ typedef /*...@abstract@*/ /*...@refcounted@*/
   */
  enum rpmTagType_e {
  /* RPM_NULL_TYPE =  0 - never been used. */
 - /* RPM_CHAR_TYPE =  1 - never been used, same as RPM_UINT8_TYPE. */
 +RPM_CHAR_TYPE =  1,
  RPM_UINT8_TYPE =  2,
  RPM_UINT16_TYPE =  3,
  RPM_UINT32_TYPE =  4,
 @@ -37,7 +37,7 @@ enum rpmTagType_e {
   /* RPM_ASN1_TYPE = 10 - never been used. */
   /* RPM_OPENPGP_TYPE= 11 - never been used. */
  };
 -#define RPM_MIN_TYPE 2
 +#define RPM_MIN_TYPE 1
  #define RPM_MAX_TYPE 9
  #define RPM_MASK_TYPE 0x

 Index: header_internal.c
 ===
 RCS file: /v/rpm/cvs/rpm/rpmdb/header_internal.c,v
 retrieving revision 1.19
 diff -p -u -w -r1.19 header_internal.c
 --- header_internal.c 31 Jul 2008 02:40:08 - 1.19
 +++ header_internal.c 18 Nov 2009 03:58:11 -
 @@ -44,11 +44,13 @@ int headerVerifyInfo(rpmuint32_t il, rpm
  for (i = 0; i  il; i++) {
   info-tag = (rpmuint32_t) ntohl(pe[i].tag);
   info-type = (rpmuint32_t) ntohl(pe[i].type);
 +#ifdef DYING
   /* XXX Convert RPMTAG_FILESTATE to RPM_UINT8_TYPE. */
   if (info-tag == 1029  info-type == 1) {
  info-type = RPM_UINT8_TYPE;
  pe[i].type = (rpmuint32_t) htonl(info-type);
   }
 +#endif
   info-offset = (rpmint32_t) ntohl(pe[i].offset);
  assert(negate || info-offset = 0); /* XXX insurance */
   if (negate)

 Checking  yes the above patch works with rpm -qavv and --rebuilddb.

rpm -qa foo\* is still broken though.. ;p


rpmdb segfaults after upgrade from rpm 4.6.0

2009-11-17 Thread Per Øyvind Karlsen
I just did a fresh install of Mandriva Linux and updated to rpm 5.2 now,
and this happens:

[r...@localhost rpm]# valgrind rpm --rebuilddb -vvv --rpmdbdebug
==23457== Memcheck, a memory error detector
==23457== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==23457== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==23457== Command: rpm --rebuilddb -vvv --rpmdbdebug
==23457==
D: pool fd: created size 896 limit -1 flags 0
D: pool ds: created size 224 limit -1 flags 0
D: pool syck:   created size 32 limit -1 flags 0
D: pool ht: created size 64 limit -1 flags 0
D: pool mire:   created size 128 limit -1 flags 0
D: pool lua:created size 56 limit -1 flags 0
D: pool ts: created size 1208 limit -1 flags 0
D: pool db: created size 304 limit -1 flags -1
== rpmdbNew(/, (null), 0x0, 0644, 0x0) db 0xb604a30
-- db 0xb604a30 ++ 1 rpmdbNew at rpmdb.c:1219
== dbiOpen(0xb604a30, Packages(0), 0x0)
D: opening  db environment /var/lib/rpm/Packages auto_commit:create
D: opening  db index   /var/lib/rpm/Packages rdonly mode=0x0
D: locked   db index   /var/lib/rpm/Packages
D: rebuilding database /var/lib/rpm into /var/lib/rpmrebuilddb.23457
D: creating directory /var/lib/rpmrebuilddb.23457
D: opening old database with dbapi 3
== rpmdbNew(/, /var/lib/rpm, 0x0, 0644, 0x2) db 0xb48a6e0
-- db 0xb48a6e0 ++ 1 rpmdbNew at rpmdb.c:1219
== dbiOpen(0xb48a6e0, Packages(0), 0x0)
D: opening  db environment /var/lib/rpm/Packages auto_commit
D: opening  db index   /var/lib/rpm/Packages rdonly mode=0x0
D: opening new database with dbapi 3
== rpmdbNew(/, /var/lib/rpmrebuilddb.23457, 0x42, 0644, 0x0) db 0xb497910
-- db 0xb497910 ++ 1 rpmdbNew at rpmdb.c:1219
== dbiOpen(0xb497910, Packages(0), 0x0)
D: opening  db environment /var/lib/rpmrebuilddb.23457/Packages
auto_commit:create
D: opening  db index   /var/lib/rpmrebuilddb.23457/Packages create
mode=0x42
== dbiOpen(0xb48a6e0, Packages(0), 0x0)
D: pool mi: created size 200 limit -1 flags 0
-- db 0xb48a6e0 ++ 2 matchIterator at rpmdb.c:2913
== dbiOpen(0xb48a6e0, Packages(0), 0x0)
D: pool dig:created size 408 limit -1 flags 0
==23457==
==23457== Process terminating with default action of signal 11 (SIGSEGV)
==23457==  Bad permissions for mapped region at address 0x403639C
==23457==at 0x531E0FD: headerVerifyInfo (header_internal.c:50)
==23457==by 0x5322EC1: headerCheck (pkgio.c:1022)
==23457==by 0x532B4A3: rpmmiNext (rpmdb.c:2605)
==23457==by 0x533039B: rpmdbRebuild (rpmdb.c:4198)
==23457==by 0x50C2680: rpmtsRebuildDB (rpmts.c:146)
==23457==by 0x404801: main (rpmqv.c:765)
==23457==
==23457== HEAP SUMMARY:
==23457== in use at exit: 190,626 bytes in 3,311 blocks
==23457==   total heap usage: 10,231 allocs, 6,920 frees, 88,236,867 bytes
allocated
==23457==
==23457== LEAK SUMMARY:
==23457==definitely lost: 0 bytes in 0 blocks
==23457==indirectly lost: 0 bytes in 0 blocks
==23457==  possibly lost: 4,529 bytes in 8 blocks

Gonna do some debugging myself now, but I figure others might have a better
idea about it already.. ;)

--
Regards,
Per Øyvind


Re: rpmdb segfaults after upgrade from rpm 4.6.0

2009-11-17 Thread Jeff Johnson

On Nov 17, 2009, at 12:40 PM, Per Øyvind Karlsen wrote:

 I just did a fresh install of Mandriva Linux and updated to rpm 5.2 now,
 and this happens:
 

Yup. Attached patch avoids the segfault:

Index: rpmdb/header_internal.c
===
RCS file: /v/rpm/cvs/rpm/rpmdb/header_internal.c,v
retrieving revision 1.19
diff -p -u -w -r1.19 header_internal.c
--- rpmdb/header_internal.c 31 Jul 2008 02:40:08 -  1.19
+++ rpmdb/header_internal.c 17 Nov 2009 17:43:21 -
@@ -47,7 +47,9 @@ int headerVerifyInfo(rpmuint32_t il, rpm
/* XXX Convert RPMTAG_FILESTATE to RPM_UINT8_TYPE. */
if (info-tag == 1029  info-type == 1) {
info-type = RPM_UINT8_TYPE;
+#ifdef DYING
pe[i].type = (rpmuint32_t) htonl(info-type);
+#endif
}
info-offset = (rpmint32_t) ntohl(pe[i].offset);
 assert(negate || info-offset = 0);   /* XXX insurance */

headerCheck is rather useless now that header blob's are PROT_READ protected.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature


Re: rpmdb segfaults after upgrade from rpm 4.6.0

2009-11-17 Thread Per Øyvind Karlsen
2009/11/17 Jeff Johnson n3...@mac.com


 On Nov 17, 2009, at 12:40 PM, Per Øyvind Karlsen wrote:

  I just did a fresh install of Mandriva Linux and updated to rpm 5.2 now,
  and this happens:
 

 Yup. Attached patch avoids the segfault:

 Index: rpmdb/header_internal.c
 ===
 RCS file: /v/rpm/cvs/rpm/rpmdb/header_internal.c,v
 retrieving revision 1.19
 diff -p -u -w -r1.19 header_internal.c
 --- rpmdb/header_internal.c 31 Jul 2008 02:40:08 -  1.19
 +++ rpmdb/header_internal.c 17 Nov 2009 17:43:21 -
 @@ -47,7 +47,9 @@ int headerVerifyInfo(rpmuint32_t il, rpm
/* XXX Convert RPMTAG_FILESTATE to RPM_UINT8_TYPE. */
if (info-tag == 1029  info-type == 1) {
info-type = RPM_UINT8_TYPE;
 +#ifdef DYING
pe[i].type = (rpmuint32_t) htonl(info-type);
 +#endif
}
info-offset = (rpmint32_t) ntohl(pe[i].offset);
  assert(negate || info-offset = 0);   /* XXX insurance */

 headerCheck is rather useless now that header blob's are PROT_READ
 protected.

 73 de Jeff

d'oh:
== dbiOpen(0xb604a30, Pubkeys(266),
0x0)
D: opening  db index   /var/lib/rpm/Pubkeys rdonly
mode=0x0
-- db 0xb604a30 ++ 2 matchIterator at
rpmdb.c:2913
== dbiOpen(0xb604a30, Packages(0),
0x0)
D: pool h:  created size 344 limit -1 flags
0
== dbiOpen(0xb604a30, Packages(0),
0x0)
-- db 0xb604a30 -- 2 rpmdbClose at
rpmdb.c:1034
D: pool iob:created size 40 limit -1 flags
0
D: == DSA pubkey id dd684d7a 26752624
(h#1)
D: rpmdb: read h# 769 Header V3 DSA signature: OK, key ID
26752624
error: rpmdb: damaged header #769 retrieved --
skipping.
D: rpmdb: read h#1538 Header V3 DSA signature: OK, key ID 26752624
error: rpmdb: damaged header #1538 retrieved -- skipping.
==740==
==740== Process terminating with default action of signal 11 (SIGSEGV)
==740==  Bad permissions for mapped region at address 0x4217000
==740==at 0x4C23414: memcpy (mc_replace_strmem.c:482)
==740==by 0x6900C8B: __db_goff_rpmdb (db_overflow.c:174)
==740==by 0x690F487: __db_ret_rpmdb (db_ret.c:46)
==740==by 0x68E7F1E: __dbc_get_rpmdb (db_cam.c:868)
==740==by 0x68F8607: __dbc_get_pp_rpmdb (db_iface.c:2095)
==740==by 0x533A718: db3cget (db3.c:765)
==740==by 0x53250F9: dbiGet (rpmdb.h:643)
==740==by 0x532AD5A: rpmmiGet (rpmdb.c:2472)
==740==by 0x532B2EF: rpmmiNext (rpmdb.c:2563)
==740==by 0x53303F7: rpmdbRebuild (rpmdb.c:4198)
==740==by 0x50C2730: rpmtsRebuildDB (rpmts.c:146)
==740==by 0x404801: main (rpmqv.c:765)
==740== Invalid free() / delete / delete[]
==740==at 0x4C21A58: free (vg_replace_malloc.c:325)
==740==by 0x5D5911A: ??? (in /lib64/libc-2.10.1.so)
==740==by 0x5D58CB1: ??? (in /lib64/libc-2.10.1.so)
==740==by 0x4A1D518: _vgnU_freeres (vg_preloaded.c:62)
==740==by 0x7FEFFF60F: ???
==740==by 0x6900C8B: __db_goff_rpmdb (db_overflow.c:174)
==740==by 0x690F487: __db_ret_rpmdb (db_ret.c:46)
==740==by 0x68E7F1E: __dbc_get_rpmdb (db_cam.c:868)
==740==by 0x68F8607: __dbc_get_pp_rpmdb (db_iface.c:2095)
==740==by 0x533A718: db3cget (db3.c:765)
==740==by 0x53250F9: dbiGet (rpmdb.h:643)
==740==by 0x532AD5A: rpmmiGet (rpmdb.c:2472)
==740==  Address 0x403b398 is not stack'd, malloc'd or (recently) free'd
==740==
==740==
==740== HEAP SUMMARY:
==740== in use at exit: 209,881 bytes in 3,346 blocks
==740==   total heap usage: 10,386 allocs, 7,044 frees, 88,845,335 bytes
allocated
==740==
==740== LEAK SUMMARY:
==740==definitely lost: 0 bytes in 0 blocks
==740==indirectly lost: 0 bytes in 0 blocks
==740==  possibly lost: 13,176 bytes in 16 blocks
==740==still reachable: 196,705 bytes in 3,330 blocks
==740== suppressed: 0 bytes in 0 blocks
==740== Rerun with --leak-check=full to see details of leaked memory
==740==
==740== For counts of detected and suppressed errors, rerun with: -v
==740== ERROR SUMMARY: 4 errors from 1 contexts (suppressed: 4 from 4)
Minnesegmentsfeil


Re: rpmdb segfaults after upgrade from rpm 4.6.0

2009-11-17 Thread Jeff Johnson

On Nov 17, 2009, at 12:49 PM, Per Øyvind Karlsen wrote:

 
 
 2009/11/17 Jeff Johnson n3...@mac.com
 
 On Nov 17, 2009, at 12:40 PM, Per Øyvind Karlsen wrote:
 
  I just did a fresh install of Mandriva Linux and updated to rpm 5.2 now,
  and this happens:
 
 
 Yup. Attached patch avoids the segfault:
 
 Index: rpmdb/header_internal.c
 ===
 RCS file: /v/rpm/cvs/rpm/rpmdb/header_internal.c,v
 retrieving revision 1.19
 diff -p -u -w -r1.19 header_internal.c
 --- rpmdb/header_internal.c 31 Jul 2008 02:40:08 -  1.19
 +++ rpmdb/header_internal.c 17 Nov 2009 17:43:21 -
 @@ -47,7 +47,9 @@ int headerVerifyInfo(rpmuint32_t il, rpm
/* XXX Convert RPMTAG_FILESTATE to RPM_UINT8_TYPE. */
if (info-tag == 1029  info-type == 1) {
info-type = RPM_UINT8_TYPE;
 +#ifdef DYING
pe[i].type = (rpmuint32_t) htonl(info-type);
 +#endif
}
info-offset = (rpmint32_t) ntohl(pe[i].offset);
  assert(negate || info-offset = 0);   /* XXX insurance */
 
 headerCheck is rather useless now that header blob's are PROT_READ protected.
 
 73 de Jeff
 d'oh:
 == dbiOpen(0xb604a30, Pubkeys(266), 0x0) 
   
 D: opening  db index   /var/lib/rpm/Pubkeys rdonly mode=0x0   
   
 -- db 0xb604a30 ++ 2 matchIterator at rpmdb.c:2913   
   
 == dbiOpen(0xb604a30, Packages(0), 0x0)  
   
 D: pool h:  created size 344 limit -1 flags 0 
   
 == dbiOpen(0xb604a30, Packages(0), 0x0)  
   
 -- db 0xb604a30 -- 2 rpmdbClose at rpmdb.c:1034  
   
 D: pool iob:created size 40 limit -1 flags 0  
   
 D: == DSA pubkey id dd684d7a 26752624 (h#1)   
   
 D: rpmdb: read h# 769 Header V3 DSA signature: OK, key ID 26752624
   
 error: rpmdb: damaged header #769 retrieved -- skipping.  
   
 D: rpmdb: read h#1538 Header V3 DSA signature: OK, key ID 26752624
 error: rpmdb: damaged header #1538 retrieved -- skipping.

Disable all the header check's.

The root issue (my guess) is likely RPM_CHAR_TYPE == 1
or not. RPM_CHAR_TYPE was removed in rpm-5.0, but
is still present in @rpm.org code.

RPM_CHAR_TYPE is used solely for keeping track of
file state after install. Its a bit mask of 8 bits, the typing
Simply Does Not Matter (but headerCheck() will fail headers)

Also see the patch I sent back in June knocking out as
many incompatibilities with rpm-4.7.x as possible.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature


Re: rpmdb segfaults after upgrade from rpm 4.6.0

2009-11-17 Thread Per Øyvind Karlsen
2009/11/17 Jeff Johnson n3...@mac.com


 On Nov 17, 2009, at 12:49 PM, Per Øyvind Karlsen wrote:



 2009/11/17 Jeff Johnson n3...@mac.com


 On Nov 17, 2009, at 12:40 PM, Per Øyvind Karlsen wrote:

  I just did a fresh install of Mandriva Linux and updated to rpm 5.2 now,
  and this happens:
 

 Yup. Attached patch avoids the segfault:

 Index: rpmdb/header_internal.c
 ===
 RCS file: /v/rpm/cvs/rpm/rpmdb/header_internal.c,v
 retrieving revision 1.19
 diff -p -u -w -r1.19 header_internal.c
 --- rpmdb/header_internal.c 31 Jul 2008 02:40:08 -  1.19
 +++ rpmdb/header_internal.c 17 Nov 2009 17:43:21 -
 @@ -47,7 +47,9 @@ int headerVerifyInfo(rpmuint32_t il, rpm
/* XXX Convert RPMTAG_FILESTATE to RPM_UINT8_TYPE. */
if (info-tag == 1029  info-type == 1) {
info-type = RPM_UINT8_TYPE;
 +#ifdef DYING
pe[i].type = (rpmuint32_t) htonl(info-type);
 +#endif
}
info-offset = (rpmint32_t) ntohl(pe[i].offset);
  assert(negate || info-offset = 0);   /* XXX insurance */

 headerCheck is rather useless now that header blob's are PROT_READ
 protected.

 73 de Jeff

 d'oh:
 == dbiOpen(0xb604a30, Pubkeys(266),
 0x0)
 D: opening  db index   /var/lib/rpm/Pubkeys rdonly
 mode=0x0
 -- db 0xb604a30 ++ 2 matchIterator at
 rpmdb.c:2913
 == dbiOpen(0xb604a30, Packages(0),
 0x0)
 D: pool h:  created size 344 limit -1 flags
 0
 == dbiOpen(0xb604a30, Packages(0),
 0x0)
 -- db 0xb604a30 -- 2 rpmdbClose at
 rpmdb.c:1034
 D: pool iob:created size 40 limit -1 flags
 0
 D: == DSA pubkey id dd684d7a 26752624
 (h#1)
 D: rpmdb: read h# 769 Header V3 DSA signature: OK, key ID
 26752624
 error: rpmdb: damaged header #769 retrieved --
 skipping.
 D: rpmdb: read h#1538 Header V3 DSA signature: OK, key ID 26752624
 error: rpmdb: damaged header #1538 retrieved -- skipping.


 Disable all the header check's.

 The root issue (my guess) is likely RPM_CHAR_TYPE == 1
 or not. RPM_CHAR_TYPE was removed in rpm-5.0, but
 is still present in @rpm.org code.

 RPM_CHAR_TYPE is used solely for keeping track of
 file state after install. Its a bit mask of 8 bits, the typing
 Simply Does Not Matter (but headerCheck() will fail headers)

 Also see the patch I sent back in June knocking out as
 many incompatibilities with rpm-4.7.x as possible.

 73 de Jeff

Well, I've tried disabling both headerVerifyInfo()  headerCheck(),  idn't
really do much about the segfault it seems.. :/

Not sure what exactly I should be looking for in the patches..
--
Regards,
Per Øyvind


Re: rpmdb segfaults after upgrade from rpm 4.6.0

2009-11-17 Thread Per Øyvind Karlsen
2009/11/18 Jeff Johnson n3...@mac.com


 On Nov 17, 2009, at 12:49 PM, Per Øyvind Karlsen wrote:


 D: == DSA pubkey id dd684d7a 26752624
 (h#1)
 D: rpmdb: read h# 769 Header V3 DSA signature: OK, key ID
 26752624
 error: rpmdb: damaged header #769 retrieved --
 skipping.
 D: rpmdb: read h#1538 Header V3 DSA signature: OK, key ID 26752624
 error: rpmdb: damaged header #1538 retrieved -- skipping.
 ==740==
 ==740== Process terminating with default action of signal 11 (SIGSEGV)
 ==740==  Bad permissions for mapped region at address 0x4217000
 ==740==at 0x4C23414: memcpy (mc_replace_strmem.c:482)
 ==740==by 0x6900C8B: __db_goff_rpmdb (db_overflow.c:174)
 ==740==by 0x690F487: __db_ret_rpmdb (db_ret.c:46)
 ==740==by 0x68E7F1E: __dbc_get_rpmdb (db_cam.c:868)
 ==740==by 0x68F8607: __dbc_get_pp_rpmdb (db_iface.c:2095)
 ==740==by 0x533A718: db3cget (db3.c:765)
 ==740==by 0x53250F9: dbiGet (rpmdb.h:643)
 ==740==by 0x532AD5A: rpmmiGet (rpmdb.c:2472)
 ==740==by 0x532B2EF: rpmmiNext (rpmdb.c:2563)
 ==740==by 0x53303F7: rpmdbRebuild (rpmdb.c:4198)
 ==740==by 0x50C2730: rpmtsRebuildDB (rpmts.c:146)
 ==740==by 0x404801: main (rpmqv.c:765)
 ==740== Invalid free() / delete / delete[]
 ==740==at 0x4C21A58: free (vg_replace_malloc.c:325)
 ==740==by 0x5D5911A: ??? (in /lib64/libc-2.10.1.so)
 ==740==by 0x5D58CB1: ??? (in /lib64/libc-2.10.1.so)
 ==740==by 0x4A1D518: _vgnU_freeres (vg_preloaded.c:62)
 ==740==by 0x7FEFFF60F: ???
 ==740==by 0x6900C8B: __db_goff_rpmdb (db_overflow.c:174)
 ==740==by 0x690F487: __db_ret_rpmdb (db_ret.c:46)
 ==740==by 0x68E7F1E: __dbc_get_rpmdb (db_cam.c:868)
 ==740==by 0x68F8607: __dbc_get_pp_rpmdb (db_iface.c:2095)
 ==740==by 0x533A718: db3cget (db3.c:765)
 ==740==by 0x53250F9: dbiGet (rpmdb.h:643)
 ==740==by 0x532AD5A: rpmmiGet (rpmdb.c:2472)
 ==740==  Address 0x403b398 is not stack'd, malloc'd or (recently) free'd
 ==740==


 (from #rpm irc)
 There is something funky here. Reverting the rpmmiGet() patch
 that is doing PROT_READ is fine for now. But if headers can
 be PROT_READ protected, there's no reason for any
 signature/digest/sanity checks on header blob's any more.

 But also try rpm -qavv --nosignature and see if that works.

 FWIW, given your /var/lib/rpm/Packages file, I'm able to do
 rpm -qavv and --rebuilddb using rpm-5.2 on F11 (and some minor
 debugging patches I can send along if needed) without difficulty.

 hth

 73 de Jeff

hmm, I might've misunderstood you..?
I provided you with the rpmdb that I had rebuilt with --rebuilddb, where
only rpm -qa foo\* didn't work.. I should've perhaps provided you the one
that resulted in segfaults with --rebuilddb etc. in stead?
If so, here it is:
http://212.251.243.147/~peroyvind/Packages-rpm-4.6.0ish.xz

--
Regards,
Per Øyvind


Re: rpmdb segfaults after upgrade from rpm 4.6.0

2009-11-17 Thread Jeff Johnson

On Nov 17, 2009, at 7:56 PM, Per Øyvind Karlsen wrote:

 
 hmm, I might've misunderstood you..?
 I provided you with the rpmdb that I had rebuilt with --rebuilddb, where only 
 rpm -qa foo\* didn't work.. I should've perhaps provided you the one that 
 resulted in segfaults with --rebuilddb etc. in stead?
 If so, here it is: http://212.251.243.147/~peroyvind/Packages-rpm-4.6.0ish.xz

OK, this rpmdb segfaults on rpm -qa like this

D: pool h:  created size 212 limit -1 flags 0
error: rpmdb: damaged header #769 retrieved -- skipping.
D: rpmdb: read h#1538 Header sanity check: OK
error: rpmdb: damaged header #1538 retrieved -- skipping.
==8128== 
==8128== Process terminating with default action of signal 11 (SIGSEGV)
==8128==  Bad permissions for mapped region at address 0x4015000
==8128==at 0x4006F64: memcpy (mc_replace_strmem.c:482)
==8128==by 0x42BE438: __db_goff_rpmdb (in /usr/lib/librpmmisc-5.2.so)
==8128==by 0x42CBB90: __db_ret_rpmdb (in /usr/lib/librpmmisc-5.2.so)
==8128==by 0x42A843D: __dbc_get_rpmdb (in /usr/lib/librpmmisc-5.2.so)
==8128==by 0x42B6ADF: __dbc_get_pp_rpmdb (in /usr/lib/librpmmisc-5.2.so)
==8128==by 0x40F8195: db3cget (db3.c:765)
==8128==by 0x40EBA86: rpmmiGet (rpmdb.h:647)
==8128==by 0x40EBDD3: rpmmiNext (rpmdb.c:2567)
==8128==by 0x408DCA3: rpmgiNext (rpmgi.c:608)
==8128==by 0x407646F: rpmgiShowMatches (query.c:396)
==8128==by 0x4077461: rpmQueryVerify (query.c:495)
==8128==by 0x4077FE6: rpmcliArgIter (query.c:770)
==8128== 

With the patch below, rpm -qavv and rpm --rebuilddb work:

@@ -2483,7 +2487,7 @@ static int rpmmiGet(dbiIndex dbi, DBC * 
vp-data = uh;
rc = dbiGet(dbi, dbcursor, kp, vp, DB_SET);
if (rc == 0) {
-   if (mprotect(uh, uhlen, PROT_READ) != 0)
+   if (mprotect(uh, uhlen, PROT_READ|PROT_WRITE) != 0)
fprintf(stderr, == mprotect(%p[%u],0x%x) error(%d): %s\n,
uh, uhlen, PROT_READ,
errno, strerror(errno));

Reverting the patch, and using the rebuilt rpmdb, rpm -qa and rpm --rebuilddb 
continue to work.

Returning to the original Packages file, and doing --rebuilddb, an
attempt to write to PROT_READ hardware protected memory causes a SIG11:

==25163== Process terminating with default action of signal 11 (SIGSEGV)
==25163==  Bad permissions for mapped region at address 0x401E39C
==25163==at 0x40E1144: headerVerifyInfo (header_internal.c:50)
==25163==by 0x40E4A0D: headerCheck (pkgio.c:1022)
==25163==by 0x40EBE98: rpmmiNext (rpmdb.c:2609)
==25163==by 0x40EF871: rpmdbRebuild (rpmdb.c:4202)
==25163==by 0x409A6A0: rpmtsRebuildDB (rpmts.c:146)
==25163==by 0x804C0E5: main (rpmqv.c:765)

Applying the patch I gave earlier, I come full cycle to the pattern of behavior 
you are reporting.

So what is needed is a --rebuilddb with the PROT_WRITE added, and
then revert to only setting PROT_READ. How you wish to achieve
that conversion on the fly is up to you. The forward looking goal
is to use PROT_READ hardware protection rather than all the
overly complex and senseless digest/signature/sanity checking
with is most definitely cpu intensive and largely unnecessary.

So headerVerifyInfo() (and the attempt to change PROT_READ memory
that results in a SIG11) is going into the bit bucket where signature/digest
checking of rpmdb Headers has already been discarded.

(aside)
There's another way to fix this issue by re-adding RPM_CHAR_TYPE rather
than attempting to change a data type in a PROT_READ header on the fly
as is being done in rpmdb/header_internal.c.

The goal in RPM-5.0 is/was to make all header data types unsigned,
and that goal was most definitely achieved, and signified by doing a major
release of RPM in which no compatibility was promised or intended.

I'm not about to go all the way back and re-release every version of RPM since 
rpm-5.0
to retrofit compatibility  for distros that don't use rpm5.org code 
particularly since this issue
was both known and discussed before rpm-5.0 was released. See rpm-devel 
archives.

But a conversion tool could be written rather easily.

The RPM_CHAR_TYPE is never found in package headers,
is used solely in one place, adding RPMTAG_FILESTATES while
installing by rpm-4.x.

Here is the patch that fixes by re-adding RPM_CHAR_TYPE . There's additional 
places needed
to remove compiler warnings (at a minimum), and likely to display RPM_CHAR_TYPE 
if
the already removed data type is to be retrofitted.

$ cvs diff rpmtag.h header_internal.c
Index: rpmtag.h
===
RCS file: /v/rpm/cvs/rpm/rpmdb/rpmtag.h,v
retrieving revision 1.62.2.2
diff -p -u -w -r1.62.2.2 rpmtag.h
--- rpmtag.h25 Jul 2009 00:56:51 -  1.62.2.2
+++ rpmtag.h18 Nov 2009 03:58:11 -
@@ -25,7 +25,7 @@ typedef /*...@abstract@*/ /*...@refcounted@*/ 
  */
 enum rpmTagType_e {
/* RPM_NULL_TYPE =  0   -