The DB back-end of my application has 2 use cases: - a normalized master DB, sharded by userid (based on their activity, not a formula such as modulo, because some users can be 1-2 order of magnitude more active than others)
- many denormalized read-only slaves, with some different models depending on what kind of queries they are serving All requests are queued in RabbitMQ, and most writes are fire-and-forget, with calculations done on the client, assuming the write worked fine and UI refreshed at most a couple of seconds later with real values. At the moment, I have total 73 tables, 432 columns and 123 relations, so not overly complex, but not a key-value store either .. Most queries are localized to sub-system which on average have 5-10 tables. So far, about 10% of traffic hit the master DB (a lot I know, this an interactive application), which is the one that really concerns me. Users make an average of 1 write request every 5 sec., so with say, with 100,000 concurrent users, that makes 20,000 tx/s. That said, that number could grow overnight and I do not want to be one more startup that redo his system under the worst of conditions as I read too often about. I got time to at least prepare a bit, without overdoing it, of course.. Sébastien On Wed, Sep 12, 2012 at 7:24 PM, John R Pierce <pie...@hogranch.com> wrote: > On 09/12/12 4:03 PM, Sébastien Lorion wrote: > >> I agree 1GB is a lot, I played around with that value, but it hardly >> makes a difference. Is there a plateau in how that value affects query >> performance ? On a master DB, I would set it low and raise as necessary, >> but what would be a good average value on a read-only DB with same spec and >> max_connections ? >> > > a complex query can require several times work_mem for sorts and hash > merges. how many queries do you expect to ever be executing > concurrently? I'll take 25% of my system memory and divide it by > 'max_connections' and use that as work_mem for most cases. > > on a large memory system doing dedicated transaction processing, I > generally shoot for about 50% of the server memory as disk cache, 1-2GB as > shared_buffers, 512MB-2GB as maintenance_work_mem, and 20-25% as work_mem > (divided by max_connections) > > > > > -- > john r pierce N 37, W 122 > santa cruz ca mid-left coast > > > > > -- > Sent via pgsql-general mailing list (pgsql-general@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/**mailpref/pgsql-general<http://www.postgresql.org/mailpref/pgsql-general> >
