Re: [GENERAL] Query caching absent "query caching"

2012-11-24 Thread Pavel Stehule
Hello you can try use plperl as cache http://okbob.blogspot.cz/2007/12/using-shared-as-table-cache-in-plperl.html Regards Pavel Stehule 2012/11/25 Bexley Hall : > Hi, > > In the absence of query caching AND NOT WANTING TO FORCE > THE APPLICATION TO DO SO EXPLICITLY, I'm looking for ideas > as

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Jeff Janes
On Wed, Nov 21, 2012 at 8:14 AM, Vlad wrote: > > > '-M prepared' produces normal results, while '-M simple' results in 40% sys > cpu. '-M extended' is somewhere in between. > I'm running it as 60 clients, 2 threads. 2 threads is pretty low for 60 clients. What happens if you increase -j to eith

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Jeff Janes
On Tue, Nov 20, 2012 at 12:00 PM, Merlin Moncure wrote: > On Tue, Nov 20, 2012 at 12:16 PM, Jeff Janes wrote: >> >> The freelist should never loop. It is written as a loop, but I think >> there is currently no code path which ends up with valid buffers being >> on the freelist, so that loop will

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Merlin Moncure
On Tue, Nov 20, 2012 at 4:08 PM, Jeff Janes wrote: >> It strikes me as cavalier to be resetting >> trycounter while sitting under the #1 known contention point for read >> only workloads. > > The only use for the trycounter is to know when to ERROR out with "no > unpinned buffers available", so no

Re: [GENERAL] Restore postgres to specific time

2012-11-24 Thread wd
On Sun, Nov 25, 2012 at 4:25 AM, Jeff Janes wrote: > On Sat, Nov 24, 2012 at 6:00 AM, wd wrote: > > Yes, you are right, after set the two command, the recovery will stop at > > that time. > > > > But there is an other question, how to make this recovered Postgres can > be > > read and write? Ac

[GENERAL] Query caching absent "query caching"

2012-11-24 Thread Bexley Hall
Hi, In the absence of query caching AND NOT WANTING TO FORCE THE APPLICATION TO DO SO EXPLICITLY, I'm looking for ideas as to how I should "future-safe" the design of some custom user base types and functions thereon. Specifically, I have several computationally expensive functions that derive t

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Gavin Flower
On 25/11/12 11:11, Kevin Grittner wrote: Gavin Flower wrote: We found that the real-world production performance of a web application servicing millions of we hits per day with thousands of concurrent users improved when we reconfigured our database connection pool to be about 35 instead of 55,

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Vlad
it's session mode and the pool size is 1200 (cause I need to grantee that in the worst case we have enough slots for all possible clients), however even at the times preceding high-cpu-sys-stall, the number postmasters are like 15-20. When stall happens, it starts to raise but that's the result of

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Kevin Grittner
Gavin Flower wrote: >> We found that the real-world production performance of a web >> application servicing millions of we hits per day with thousands >> of concurrent users improved when we reconfigured our database >> connection pool to be about 35 instead of 55, on a 16 core box >> with a 40 d

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Vlad
> what pgbouncer mode, and how large is your pool. > > '-M prepared' produces normal results, while '-M simple' results in 40% sys cpu. '-M extended' is somewhere in between. I'm running it as 60 clients, 2 threads. -- Vlad

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Merlin Moncure
On Wed, Nov 21, 2012 at 11:05 AM, Vlad wrote: > it's session mode and the pool size is 1200 (cause I need to grantee that in > the worst case we have enough slots for all possible clients), however even > at the times preceding high-cpu-sys-stall, the number postmasters are like > 15-20. When sta

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Gavin Flower
On 25/11/12 09:30, Kevin Grittner wrote: Vlad wrote: it's session mode and the pool size is 1200 (cause I need to grantee that in the worst case we have enough slots for all possible clients), We found that the real-world production performance of a web application servicing millions of we hit

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Kevin Grittner
Vlad wrote: > it's session mode and the pool size is 1200 (cause I need to > grantee that in the worst case we have enough slots for all > possible clients), We found that the real-world production performance of a web application servicing millions of we hits per day with thousands of concurrent

Re: [GENERAL] Restore postgres to specific time

2012-11-24 Thread Jeff Janes
On Sat, Nov 24, 2012 at 6:00 AM, wd wrote: > Yes, you are right, after set the two command, the recovery will stop at > that time. > > But there is an other question, how to make this recovered Postgres can be > read and write? According to the manual, Postgres should be rename > recovery.conf to

[GENERAL] How and for what to use SP-GIST?

2012-11-24 Thread Christian Hammers
Hello The release notes of PostgreSQL 9.2 praise "SP-GIST" as a major feature but I cannot find any information on how to actually use it. The main documentation seems unusual low level and the examples chapter just refers to the source code. The only examples I can find there are the regression

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Vlad
nothing changes if I increase number of threads. pgbouncer doesn't change much. also, I think the nature of high-sys-cpu during stall and and when I run pgbench is different. During pgbench it's constantly at 30-40%, while during stall it sits at low 5-15% and then spikes to 90% after a while, wit

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Merlin Moncure
On Wed, Nov 21, 2012 at 10:43 AM, Jeff Janes wrote: > On Wed, Nov 21, 2012 at 7:29 AM, Vlad Marchenko wrote: > >> update on my problem: despite pgbouncer, the problem still occures on my >> end. > > As Merlin asked, how big is the pool? Maybe you are using a large > enough pool so as to defeat t

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Jeff Janes
On Wed, Nov 21, 2012 at 9:05 AM, Vlad wrote: > it's session mode and the pool size is 1200 (cause I need to grantee that in > the worst case we have enough slots for all possible clients), Wouldn't the clients prefer to wait 100ms to get a connnection if that means their query finishes in 100ms,

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Merlin Moncure
On Tue, Nov 20, 2012 at 12:16 PM, Jeff Janes wrote: > On Tue, Nov 20, 2012 at 9:05 AM, Merlin Moncure wrote: >> On Tue, Nov 20, 2012 at 10:50 AM, Jeff Janes wrote: >>> >>> I wouldn't expect so. Increasing shared_buffers should either fix >>> free list lock contention, or leave it unchanged, not

Re: [GENERAL] High SYS CPU - need advise

2012-11-24 Thread Jeff Janes
On Tue, Nov 20, 2012 at 9:05 AM, Merlin Moncure wrote: > On Tue, Nov 20, 2012 at 10:50 AM, Jeff Janes wrote: >> >> I wouldn't expect so. Increasing shared_buffers should either fix >> free list lock contention, or leave it unchanged, not make it worse. > > AIUI, that is simply not true (unless y

Re: [GENERAL] 9.1 to 9.2 requires a dump/reload?

2012-11-24 Thread Bruce Momjian
On Sat, Nov 24, 2012 at 06:03:32PM +0100, Christian Hammers wrote: > Hello > > Can you remember where did you read that? There is no mention of GIST on > http://www.postgresql.org/docs/9.2/static/upgrading.html and a database > which uses GIST indexes *seems* to work just finde after upgrading wit

Re: [GENERAL] 9.1 to 9.2 requires a dump/reload?

2012-11-24 Thread Christian Hammers
Hello Can you remember where did you read that? There is no mention of GIST on http://www.postgresql.org/docs/9.2/static/upgrading.html and a database which uses GIST indexes *seems* to work just finde after upgrading with pg_upgrade. bye, -christian- Am Mon, 22 Oct 2012 15:02:13 -0700 schrieb

Re: [GENERAL] ERROR: query has no destination for result data

2012-11-24 Thread Adrian Klaver
On 11/24/2012 12:46 AM, Gavan Schneider wrote: On Friday, November 23, 2012 at 21:36, Peter Kroon wrote: Hello, I wish to return the SELECT statement. Ho can I achieve this? DO $$ DECLARE v_some_id int=14; BEGIN /* more queries here...

Re: [GENERAL] sefety of passwords for web-service applications

2012-11-24 Thread Rafal Pietrak
On Sat, 2012-11-24 at 01:41 -0800, Chris Travers wrote: [-] > I think the closest you can come are the new security barriers in > views. Yes. This is actually what I'm currently thinking to use. A security definer function, invoked within a WITH clausure of a VIEW. -R --

Re: [GENERAL] Restore postgres to specific time

2012-11-24 Thread wd
Yes, you are right, after set the two command, the recovery will stop at that time. But there is an other question, how to make this recovered Postgres can be read and write? According to the manual, Postgres should be rename recovery.conf to recovery.done, but it didn't. I've tried pg_ctl promo

Re: Partial authentication (was Re: [GENERAL] sefety of passwords for web-service applications)

2012-11-24 Thread Chris Travers
On Sat, Nov 24, 2012 at 3:37 AM, Chris Angelico wrote: > On Sat, Nov 24, 2012 at 8:41 PM, Chris Travers > wrote: > > 2) PostgreSQL allows you to move this authentication to a secondary > service > > like Kerberos, LDAP, or anything PAM supported. This means that if you > want > > to you can us

Re: [GENERAL] sefety of passwords for web-service applications

2012-11-24 Thread Bill Moran
On Sat, 24 Nov 2012 11:05:38 +0100 "Vlad K." wrote: > > On 11/24/2012 10:15 AM, Rafal Pietrak wrote: > > Some improvement in passwords safety could be gained, if the database > > table access methods (e.g. SELECT...) provided means to limit that > > access to just one entry at a time, and return

Re: [GENERAL] alter sequence

2012-11-24 Thread Pavel Stehule
Hello you can't to do it directly. Expression cannot be used instead object name you can use a plpgsql DO in this moment DO $$ BEGIN EXECUTE format('ALTER SEQUENCE %I RESTART WITH 1', pg_get_serial_sequence('table', 'id')); END; $$ LANGUAGE plpgsql; Regards Pavel Stehule 2012/11/24 Peter Kr

Partial authentication (was Re: [GENERAL] sefety of passwords for web-service applications)

2012-11-24 Thread Chris Angelico
On Sat, Nov 24, 2012 at 8:41 PM, Chris Travers wrote: > 2) PostgreSQL allows you to move this authentication to a secondary service > like Kerberos, LDAP, or anything PAM supported. This means that if you want > to you can use a dedicated password store for the passwords which is not > accessibl

[GENERAL] alter sequence

2012-11-24 Thread Peter Kroon
ALTER SEQUENCE (select pg_get_serial_sequence('table', 'id')) RESTART WITH 1; The query fails: ALTER SEQUENCE (select pg_get_serial_sequence('table... it's because of the ( I guess I'm using the wrong syntax. It is possible this way? Would be great! Best, Peter

Re: [GENERAL] sefety of passwords for web-service applications

2012-11-24 Thread Vlad K.
On 11/24/2012 10:15 AM, Rafal Pietrak wrote: Some improvement in passwords safety could be gained, if the database table access methods (e.g. SELECT...) provided means to limit that access to just one entry at a time, and return results only when (password) column hash was equal for a single ent

Re: [GENERAL] sefety of passwords for web-service applications

2012-11-24 Thread Chris Travers
On Sat, Nov 24, 2012 at 1:15 AM, Rafal Pietrak wrote: > Hello, > > I'm analysing a way to avoid one of the password attack vectors for web > services, which goes like this: > 1. acquire passwords database (assuming passwords are hashed) > 2. run cracking software on the hashes as long as you like.

[GENERAL] sefety of passwords for web-service applications

2012-11-24 Thread Rafal Pietrak
Hello, I'm analysing a way to avoid one of the password attack vectors for web services, which goes like this: 1. acquire passwords database (assuming passwords are hashed) 2. run cracking software on the hashes as long as you like. Obviously the attack is more difficult if the step-1 is made as

Re: [GENERAL] ERROR: query has no destination for result data

2012-11-24 Thread Gavan Schneider
On Friday, November 23, 2012 at 21:36, Peter Kroon wrote: Hello, I wish to return the SELECT statement. Ho can I achieve this? DO $$ DECLARE v_some_id int=14; BEGIN /* more queries here... */ SELECT 'this is text'; END