Hi!
I had the feeling that the yum sqlite db is slow - too slow.
I wrote a small python program that shows the problem: Joining with packages
to map the pkgId to the pkgKey is expensive.
The values strongly differ depending on the presence of indices (surprise!).
So my program runs the test twice and tries to create/drop the index. Write
access is needed for altering the index.
Output on my computer (FC6 core) is:
Reading requires.name for pkg 'glibc'
Access via pkgId: 7.07 0.0707
Access via pkgKey: 1.39 0.0139
Access via pkgKey is 5.1 times faster
creating index took 0.38s
Access via pkgId: 2.21 0.0221
Access via pkgKey: 0.01 0.0001
Access via pkgKey is 212.7 times faster
The first number is the overall time and the second the time per select.
have fun
Florian
#!/usr/bin/python
import sqlite, time
def runcheck(cursor, sql, msg):
nr = 100
t1 = time.time()
for i in xrange(nr):
cursor.execute(sql)
cursor.fetchall()
t2 = (time.time()-t1)
print msg % (t2, t2/nr)
return t2
def main():
db = sqlite.connect("/var/cache/yum/core/primary.xml.gz.sqlite")
c = db.cursor()
print "Reading requires.name for pkg 'glibc'"
c.execute('SELECT pkgKey, pkgId FROM packages WHERE name="glibc"')
pkgKey, pkgId = c.fetchone()
keySql = 'SELECT requires.name FROM requires WHERE pkgKey="%s"' % pkgKey
keyMsg = "Access via pkgKey: %5.2f %6.4f"
idSql = 'SELECT requires.name FROM requires, packages WHERE requires.pkgKey=packages.pkgKey and pkgId="%s"' % pkgId
idMsg = "Access via pkgId: %5.2f %6.4f"
try:
c.execute("DROP INDEX IF EXISTS pkgrequires")
except sqlite.DatabaseError:
print "Failed to remove index. Will run with index"
idTime = runcheck(c, idSql, idMsg)
keyTime = runcheck(c, keySql, keyMsg)
print "Access via pkgKey is %.1f times faster" % (idTime/keyTime)
try:
t1 = time.time()
c.execute("CREATE INDEX pkgrequires on requires (pkgKey)")
print "Creating index took %.2fs" % (time.time()-t1)
except sqlite.DatabaseError:
print "Failed to create index. Will run without index"
idTime = runcheck(c, idSql, idMsg)
keyTime = runcheck(c, keySql, keyMsg)
print "Access via pkgKey is %.1f times faster" % (idTime/keyTime)
main()
_______________________________________________
Yum-devel mailing list
[email protected]
https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
