can you guys helpe me out in designing cache structures/ hash tables for these cases
1. Scenario One:
* Huge number of reads
* Few writes
* One row accessed from db per request
* All data written to db is used.
* Medium number of records.
* Huge number of reads
* Few writes
* One row accessed from db per request
* All data written to db is used.
* Medium number of records.
Since the large number of read need to be fast I chose this design:
A hashtable caches records. They are cleared based on expiration time
and limit on maximum number of cached records. When a record is updated
in the db, it is deleted from the cache and recached next time it is
queried. This part could be handled by implementation you linked to.
and limit on maximum number of cached records. When a record is updated
in the db, it is deleted from the cache and recached next time it is
queried. This part could be handled by implementation you linked to.
A persistent background thread periodically flushes the changed/new
values in the cache to the database. Additional calculations could be
added here to chose the optimal cache entrees to delete or precache.
values in the cache to the database. Additional calculations could be
added here to chose the optimal cache entrees to delete or precache.
2. Scenario Two:
* Few reads
* Huge number of writes
* Many rows accessed from db per request
* Most data written to db is discarded.
* Huge number of small records.
* Few reads
* Huge number of writes
* Many rows accessed from db per request
* Most data written to db is discarded.
* Huge number of small records.
This is the strangest so far. Because of the huge number of records,
most of which will just be discarded anyway, I try to keep away from
the database as much as possible. When it's time to flush them to the
db I aggregate the records into groups, cpickle, and lzo them. I am
still playing with this but I notice a good performance improvement
already.
most of which will just be discarded anyway, I try to keep away from
the database as much as possible. When it's time to flush them to the
db I aggregate the records into groups, cpickle, and lzo them. I am
still playing with this but I notice a good performance improvement
already.
3. Scenario Three:
* Huge number of writes
* Huge number of reads
* One row accessed from db per request
* Small number of records.
This is the same as the first except I have space to cache everything. * Huge number of writes
* Huge number of reads
* One row accessed from db per request
* Small number of records.
This means I can get rid of the time() check for each cache check.
Stay in the know. Pulse on the new Yahoo.com. Check it out.
_______________________________________________ Tutor maillist - [email protected] http://mail.python.org/mailman/listinfo/tutor
