JD> hassles. I don't really know anything about BDB, but I'm surprised the JD> performance of a properly indexed SQL database can't get close to it.
BDB works directly with files, but SQL backend working with PostgreSQL or database like this needs to send queries over network -- that has significant overhead, something like millisecond or so. this practically means you cannot make more than few thousands queries per second, that is quite prohibitive. this problem does not exist when you use SQL database directly -- people write complex queries with it and retrieve exactly what do they want in one pass. but we have to do query on each slot access, that's much worse.. in db-postmodern backend we've implemented btree lisp-side cache for this -- it can reduce query count greatly, but of course it works only on some workloads. JD> If the performance with SQLite is poor, has it been verified to be JD> only a SQLite issue and not something related to CLSQL? OTOH SQLite does not have this problem. that's why it can be much faster than PostgreSQL. however, as far as i remember, elephant's code that works with CLSQL works in extremely non-optimal way in some cases -- to retrieve single value from index via cursor, it retrieves all contents of index, sorts it, and retrieves that single value. but if application does not need range queries, this might work just fine, since some time ago we've optimized get-instance-by-value to get value with direct query to btree. so i'd advice to do benchmarking before coming to conclusions. if db-clsql + sqlite will work fine for given size of data base and queries, why should one need something else? _______________________________________________ elephant-devel site list elephant-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/elephant-devel
