Re: [PERFORM] JDBC question for PG 8.3.9

2010-04-15 Thread Dave Cramer
On Wed, Apr 14, 2010 at 7:10 PM, Craig Ringer cr...@postnewspapers.com.au wrote: On 15/04/10 04:49, Dave Crooke wrote: Hi foilks I am using PG 8.3 from Java. I am considering a performance tweak which will involve holding about 150 java.sql.PreparedStatment objects open against a single

[PERFORM] 8.3.9 - latency spikes with Linux (and tuning for consistently low latency)

2010-04-15 Thread Marinos Yannikos
Hi, we are seeing latency spikes in the 2-3 second range (sometimes 8-10s) for queries that usually take 3-4ms on our systems and I am running out of things to try to get rid of them. Perhaps someone here has more ideas - here's a description of the systems and what I've tried with no impact

Re: [PERFORM] 8.3.9 - latency spikes with Linux (and tuning for consistently low latency)

2010-04-15 Thread Tom Lane
Marinos Yannikos m...@geizhals.at writes: we are seeing latency spikes in the 2-3 second range (sometimes 8-10s) for queries that usually take 3-4ms on our systems and I am running out of things to try to get rid of them. Have you checked whether the spikes correlate with checkpoints?

Re: [PERFORM] 8.3.9 - latency spikes with Linux (and tuning for consistently low latency)

2010-04-15 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Have you checked whether the spikes correlate with checkpoints? Turn on log_checkpoints and watch for awhile. If so, fooling with the checkpoint parameters might give some relief. If that by itself doesn't do it, I've found that making the background

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Alvaro Herrera
Josh Berkus wrote: Basically, vacuuming of a table which normally takes about 20 minutes interactively with vacuum_cost_delay set to 20 had not completed after 14 hours. When I trussed it, I saw activity which indicated to me that autovacuum was doing a pollsys, presumably for cost_limit,

[PERFORM] Re: HELP: How to tame the 8.3.x JDBC driver with a biq guery result set

2010-04-15 Thread Dave Crooke
I have followed the instructions below to no avail any thoughts? http://jdbc.postgresql.org/documentation/83/query.html#query-with-cursor This is what happens when I reduce the fetch_size to 50 ... stops after about 950msec and 120 fetches (6k rows) 13:59:56,054 [PerfDataMigrator]

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: We don't call pollsys anywhere. Something in Solaris must be doing it under the hood. pg_usleep calls select(), and some googling indicates that select() is implemented as pollsys() on recent Solaris versions. So Josh's assumption that those

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: We don't call pollsys anywhere. Something in Solaris must be doing it under the hood. pg_usleep calls select(), and some googling indicates that select() is implemented as pollsys() on recent Solaris versions. So Josh's

Re: [PERFORM] stats collector suddenly causing lots of IO

2010-04-15 Thread Tom Lane
Chris li...@deksai.com writes: I have a lot of centos servers which are running postgres. Postgres isn't used that heavily on any of them, but lately, the stats collector process keeps causing tons of IO load. It seems to happen only on servers with centos 5. The versions of postgres that

Re: [PERFORM] 8.3.9 - latency spikes with Linux (and tuning for consistently low latency)

2010-04-15 Thread Greg Smith
Marinos Yannikos wrote: vm.dirty_ratio = 80 This is tuned the opposite direction of what you want. The default tuning in the generation of kernels you're using is: /proc/sys/vm/dirty_ratio = 10 /proc/sys/vm/dirty_background_ratio = 5 And those should be considered upper limits if you want

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: But it shouldn't be sleeping after each page with normal cost_delay parameters, should it? Right, that's why I find this puzzling. If the problem was easier to reproduce it would be easier to analyze. The behavior would be explained if VacuumCostLimit

Re: [PERFORM] Autovaccum with cost_delay does not complete on one solaris 5.10 machine

2010-04-15 Thread Josh Berkus
Josh, are you sure that both servers are identical in terms of both GUC-related and per-table autovacuum settings? I should check per-table. GUC, yes, because the company has source management for config files. -- -- Josh Berkus

[PERFORM] SOLVED: Re: HELP: How to tame the 8.3.x JDBC driver with a biq guery result set

2010-04-15 Thread Dave Crooke
When a connection is used for both reading and writing, a commit() also destroys any open cursors. Simple workaround - use two connections. See full discussion on JDBC list. Cheers Dave On Thu, Apr 15, 2010 at 3:01 PM, Dave Crooke dcro...@gmail.com wrote: I have followed the instructions