rpm -qa performance with GPG key(s) installed in the DB

2009-11-30 Thread Joshua Burns
Hi, 

Just got rpm 5.1.9 compiled on Solaris, and have noticed that, after GPG keys 
are installed in the RPM database, rpm -qa performance slows noticeably. 
Cryptosignatures are a key element of our plans for packaging, so forgoing them 
would be a pretty big issue. 

Any thoughts? 



  

Re: rpm -qa performance with GPG key(s) installed in the DB

2009-11-30 Thread Jeff Johnson

On Nov 30, 2009, at 5:17 PM, Joshua Burns wrote:

 Hi, 
 
 Just got rpm 5.1.9 compiled on Solaris, and have noticed that, after GPG keys 
 are installed in the RPM database, rpm -qa performance slows noticeably. 
 Cryptosignatures are a key element of our plans for packaging, so forgoing 
 them would be a pretty big issue. 
 
 Any thoughts? 
 


The terms key element  and forgoing seem to be at odds with each other.

I'm not sure whether you want fast crypto or no crypto.

So here's both answers, as well as the development answer:

FAST CRYPTO


rpm has _THREE_ crypto implementations (if built that way):
BeeCrypt
NSS
OpenSSL

They are selectable with
--usecrypto bc
--usecrypto nss
--usecrypto openssl

RPM is also insturmented with its own benchmarking using --stats.

Build, measure, use fastest.

For extra credit, try using callgrind. BeeCrypt is 10-15% faster.


NO CRYPTO
===

Otherwise, one can disable signature/digest checking persistently on rpm -qa 
using

   Verify digest/signature flags for various rpm modes:
#   0x30300 (_RPMVSF_NODIGESTS)--nohdrchk  if set, don't check 
digest(s)
#   0xc0c00 (_RPMVSF_NOSIGNATURES) --nosignature   if set, don't check 
signature(s)
#   0xf (_RPMVSF_NOPAYLOAD)--nolegacy  if set, check 
header+payload (if possible)
#   0x00f00 (_RPMVSF_NOHEADER) --nohdrchk  if set, don't check 
rpmdb headers
#
#   For example, the value 0xf0c00 (=0xf+0xc0c00) disables legacy
#   digest/signature checking, disables signature checking, but attempts
#   digest checking, also when retrieving headers from the database.
#
#   The checking overhead was ~11ms per header for digests/signatures on
#   a 600 Mhz Dell SMP server circa 1998.
#
#   Each header from the database is checked only when first encountered
#   for each database open.
#
#   Note: the %_vsflags_erase applies to --upgrade/--freshen modes as
#   well as --erase.
#
%__vsflags  0xf
%_vsflags_build %{__vsflags}
%_vsflags_erase %{__vsflags}
%_vsflags_install   %{__vsflags}
%_vsflags_query %{__vsflags}
%_vsflags_rebuilddb %{__vsflags}
%_vsflags_verify%{__vsflags}


NO CRYPTO NEEDED
==

Note that rpm on cvs HEAD no longer bothers with digest/signature checks
on rpmdb Headers. Its kinda pointless to verify memory that is PROT_READ
protected using mmap(2) (as on cvs HEAD).

The signature needs to be verified only when installing.

So far rpm -qa is merely 3x faster:
 $ /usr/bin/time rpm -qa  /dev/null
 0.00user 0.58system 0:00.66elapsed 88%CPU (0avgtext+0avgdata 0maxresident)k
 0inputs+0outputs (0major+63810minor)pagefaults 0swaps

And I expect faster yet when headerLoad() is eliminated, and rpm -qa reads 
installed
package names solely from an rpmdb table.

Note that Berkeley DB will do sha1 digest checking if one _REALLY_
needs that level of integrity checking on data elements.

73 de Jeff