Hi,
When we rebuild the hash in the RAM it takes too much time.
Other questions, my collegues will answer.
Murali
Differentiated Software Solutions Pvt. Ltd.
176, Ground Floor, 6th Main,
2nd Block, RT Nagar
Bangalore - 560032
Phone : 91 80 3431470
www.diffs-india.com
----- Original Message -----
From: Perrin Harkins <[EMAIL PROTECTED]>
To: Differentiated Software Solutions Pvt. Ltd <[EMAIL PROTECTED]>
Cc: <[EMAIL PROTECTED]>
Sent: Thursday, November 09, 2000 12:19 AM
Subject: Re: Fast DB access
> "Differentiated Software Solutions Pvt. Ltd" wrote:
> > 3. We have a problem rebuilding this database in the ram.... even say
every
> > 1000 requests.
>
> What problem are you having with it?
>
> > We tried using dbm and found it a good compromise solution.
> > We found that it is about 8 times faster than postgres querying.
>
> Some dbm implementations are faster than others. Depending on your data
> size, you may want to try a couple of them.
>
> > 4. Another surprising finding was.... we built a denormalised db on the
> > Linux file system itself, by using the directory and file name as the
key on
> > which we wanted to search. We found that dbm was faster than this.
>
> Did you end up with a large number of files in one directory? When
> using the file system in this way, it's a common practice to hash the
> key you're using and then split that across multiple directories to
> prevent too many files from building up in one and slowing things down.
>
> For example:
>
> "my_key" --> "dHodeifehH" --> /usr/local/data/dH/odeifehH
>
> Also, you could try using mmap for reading the files, or possibly the
> Cache::Mmap module.
>
> > We're carrying out more tests to see how scaleable is dbm.
>
> If you're using read-only data, you can leave the dbm handles persistent
> between connections. That will speed things up.
>
> You could look at BerkeleyDB, which has a built-in shared memory buffer
> and page-level locking.
>
> You could also try IPC::MM, which offers a shared memory hash written in
> C with a perl interface.
>
> > Hope these findings are useful to others.
>
> They are. Keep 'em coming.
>
> - Perrin
