Hi Ram, CBD or Berkely DB is optimized for reading, it is meant to deal with large volumes of simple non-relational basic data, so no images blobs, globs or any of that stuff... besides that it will be fast very very fast.
To give you an example in the past I used to work on a Berkely DB driven DHCP server for a ISP. The system held some 1.2 million leases for customer systems and their modems. The system had a response times in the sub millisecond range for every query, of course this was while being written to by the backend systems provisioning new customers and updating others with new leases or new subscription types. With 10M records you might not find the response time a little slower (not sure never tried that) but with todays systems and the amount of memory available I have a feeling you will not really notice the response times drop to much (if at all) Anyway, another option might be to do all of this in memory and rather then spend your time messing about with disks or even SSD's you could simply read the CBD once into memory and then work from there. I would not start of with that only bother with something like that if you get in trouble with response times, but it will certainly help a awful lot. As a last point have a look at the internals of the CBD module you are using, if this is pure Perl you will most likely want to redo that in a compiled language like C/C++, just to get that bit extras speed out of it. I hope that helps, even though there is no real Perl in here my mail I think it can still be useful. Regards, Rob On Mon, Apr 4, 2011 at 8:47 AM, Ramprasad Prasad <ramprasad...@gmail.com>wrote: > I have a CDB database with 1 Million to 10 Million records .. This is just > a > key where the app needs to lookup if an entry exists or not. I dont need > the value. My app will be looking up the key_exists() almost 100 times a > second > > What is the fastest way of implementin this > > I have a few questions > > 1) Is CDB the best way of storing such data > 2) Is CDB_File optimized for readonly lookups > 3) Is it recommended to have an in memory hash .. If memory is not an issue > ? > > > -- > Thanks > Ram > <http://www.netcore.co.in/> > > > > > n <http://pragatee.com> >
