I forgot to add one more piece of information.. I also tried the same
test with 64-bit postgresql with 6GB shared_buffers and results are the
same it drops around the same point which to me sounds like a bottleneck..
More later
-Jignesh
Jignesh K. Shah wrote:
Awww Josh,
I was just enjoying the chat on the picket fence! :-)
Anyway the workload is mixed (reads,writes) with simple to medium
queries. The workload is known to scale well. But inorder to provide
substantial input I am still trying to eliminate things that can
bottleneck. Currently I have eliminated CPU (plenty free) , RAM
(memory is 48GB RAM in this server for a 32-bit postgresql
instance), IO Storage (used the free ram to do /tmp database to
eliminate IO) and am still trying to eliminate any network
bottlenecks to say for sure we have a problem in PostgreSQL. But yes
till that final thing is confirmed (network which can very well be the
case) it could be a problem somewhere else. However the thing that
worries me is more of the big drop instead of remaining constant out
there..
Anyway more on this within a day or two once I add more network nics
between the systems to eliminate network problems (even though stats
dont show them as problems right now) and also reduce malloc lock
penalties if any.
As for other questions:
max_locks_per_transactions is set to default (10 I believe) increasing
it still seems to degrade overall throughput number.
max_connections is set to 1500 for now till I get decent scaling till
1400-1500 users.
There are no hard failures reported anywhere. Log min durations does
show that queries are now slowing down and taking longer.
OS is not swapping and also eliminated IO by putting the whole
database on /tmp
So while I finish adding more network connections between the two
systems (need to get cards) do enjoy the following URL :-)
http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-20070703-00073.html
Of course all postgresql.conf still remains from the old test so no
flames on that one again :-)
Regards,
Jignesh
Josh Berkus wrote:
Tom,
Well, if the load is a lot of short writing transactions then you'd
expect the throughput to depend on how fast stuff can be pushed down to
WAL. What have you got wal_buffers set to? Are you using a commit
delay? What's the I/O system anyway (any BB write cache on the WAL
disk?) and what wal sync method are you using?
You know, I think Jignesh needs to me on this list so I can stop
relaying questions on a workload I didn't design. Let me get him.
---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
