Re: [gentoo-portage-dev] SQLite backend?
On Tue, Jan 17, 2006 at 02:01:29AM -0200, Gustavo Sverzut Barbieri wrote: Hello, I admit I have not followed last threads about cache and new infrastructure (plugins and stuff). Might suggest you take a look at the cache rewrite- it already has a sqlite backend in it, although that's not coded against pysqlite2 (thus it needs some param work). However I followed the template and coded a SQLite3 (pysqlite2) backend: http://www.gustavobarbieri.com.br/gentoo/portage_db_sqlite.py Really should fix the 403 if you're looking for comments. Guessing this is against stable... in which case I'd point you at the 2.1 line, since the stable cache is now dead from a release standpoint (it'll exist in 2.0.*, but everything beyond is the new cache). The main problem is that it's really slow to search and calculare dependencies. Could some developer take a look and tell me where I can improve it? That's due to portage access patterns- select should be slower then flat_list/flat_hash, and the way portage pulls from the db (namely, key by key) doesn't work incredibly well for an rdbms backend- basically, the api is such that instead of doing a single select, it'll do multiple. In an ideal world, I wouldn't have to pickle/unpickle data but have its data in columns instead... one would re-encode the version strings using a pattern that is comparable by string, so you should be able to search for a given string or higher versions. Cache rewrite is intended to work towards that- still requires the restriction framework I've got in savior, but gradually working towards that. ~harring pgpx0u8c3voRz.pgp Description: PGP signature
Re: [gentoo-portage-dev] SQLite backend?
Gustavo Sverzut Barbieri wrote: Hello, I admit I have not followed last threads about cache and new infrastructure (plugins and stuff). However I followed the template and coded a SQLite3 (pysqlite2) backend: http://www.gustavobarbieri.com.br/gentoo/portage_db_sqlite.py $ env -i wget http://www.gustavobarbieri.com.br/gentoo/portage_db_sqlite.py; --10:03:17-- http://www.gustavobarbieri.com.br/gentoo/portage_db_sqlite.py = `portage_db_sqlite.py' Resolving www.gustavobarbieri.com.br... 200.210.52.20 Connecting to www.gustavobarbieri.com.br|200.210.52.20|:80... connected. HTTP request sent, awaiting response... 403 Forbidden 10:03:18 ERROR 403: Forbidden. The main problem is that it's really slow to search and calculare dependencies. Could some developer take a look and tell me where I can improve it? In an ideal world, I wouldn't have to pickle/unpickle data but have its data in columns instead... one would re-encode the version strings using a pattern that is comparable by string, so you should be able to search for a given string or higher versions. just random thoughts: you need a special number, 0 .. infinite, NULL because 1.0 is different from 1.0.0 in version numbers, the comparison functions may work in a not so intuiteve manner, take a look at versionator.eclass to see how it's handled there. version char(130) revision char(12) where version is 10 * 13 fields, every field is: 1st char: CodeSuffix Meaning 2_alpha Alpha release (earliest) 3_beta Beta release 4_pre Pre release 5_rc Release candidate 6(no suffix) Normal release 7_p Patch level blank if not applicable remaining 12 chars, version, 0 padded blank if not applicable regards -- gentoo-portage-dev@gentoo.org mailing list
