Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2015-02-12 Thread Michael Paquier
On Wed, Dec 17, 2014 at 4:55 PM, Simon Riggs wrote: > On 12 December 2014 at 03:31, Simon Riggs wrote: > > > Also attached is a new parameter called enable_sortedpath which can be > > used to turn on/off the sorted path generated by the planner. > > Now with attachment. (Thanks Jeff!) > Moved t

Re: [PERFORM] query - laziness of lateral join with function

2015-02-12 Thread Paul Callaghan
On Feb 12, 2015 9:17 PM, "Tom Lane" wrote: > The planner might produce such a result if there's an opportunity > to perform the sorting via an index on "alpha" (ie, the ORDER BY > matches some index). If it has to do an explicit sort it's gonna > do the join first. > > (If you have such an index

Re: [PERFORM] Configuration tips for very large database

2015-02-12 Thread Claudio Freire
On Thu, Feb 12, 2015 at 7:38 PM, Kevin Grittner wrote: > Nico Sabbi wrote: > >> Queries get executed very very slowly, say 20 minutes. > >> I'd like to know if someone has already succeeded in running >> postgres with 200-300M records with queries running much faster >> than this. > > If you go t

Re: [PERFORM] Configuration tips for very large database

2015-02-12 Thread Mathis, Jason
I can't speak to the numbers postgresql can or cannot do but the numbers above sound very very doable. If you can get a hold of *greg smith's postgresql high performance*, I always liked his method of tuning buffers and checkpoints using the background writer stats. All of which can help with the I

Re: [PERFORM] Survey: Max TPS you've ever seen

2015-02-12 Thread Mark Kirkwood
On 13/02/15 00:20, Gudmundsson Martin (mg) wrote: Hi all! - checkpoint_segments 1000 - checkpoint_completion_target 0.9 - wal_buffers 256MB - shared_buffers 31 gb - max_connections 500 I see that some of you are using wal_buffers = 256MB. I was under the impression that Postgres will not ben

Re: [PERFORM] Configuration tips for very large database

2015-02-12 Thread Kevin Grittner
Nico Sabbi wrote: > Can you give any hint on the configuration and on the underlying > hardware? Well, this particular web site has millions of hits per day (running up to about 20 queries per hit) from thousands of concurrent web users, while accepting logical replication from thousands of

Re: [PERFORM] Configuration tips for very large database

2015-02-12 Thread Nico Sabbi
On 02/12/2015 11:38 PM, Kevin Grittner wrote: If you go to the http://wcca.wicourts.gov/ web site, bring up any case, and click the "Court Record Events" button, it will search a table with hundreds of millions of rows. The table is not partitioned, but has several indexes on it which are usefu

Re: [PERFORM] Configuration tips for very large database

2015-02-12 Thread Kevin Grittner
Nico Sabbi wrote: > Queries get executed very very slowly, say 20 minutes. > I'd like to know if someone has already succeeded in running > postgres with 200-300M records with queries running much faster > than this. If you go to the http://wcca.wicourts.gov/ web site, bring up any case, and cl

[PERFORM] Configuration tips for very large database

2015-02-12 Thread Nico Sabbi
Hello, I've been away from postgres for several years, so please forgive me if I forgot nearly everything:-) I've just inherited a database collecting environmental data. There's a background process continually inserting records (not so often, to say the truth) and a web interface to query

Re: [PERFORM] query - laziness of lateral join with function

2015-02-12 Thread David G Johnston
Tom Lane-2 wrote > paulcc < > paulcc.two@ > > writes: >>select count(alpha.id) >>from alpha >>cross join lateral some_function(alpha.id) as some_val >>where alpha.test > >>Here the function is strict, and moreover its argument will never >>be null - hence there should al

Re: [PERFORM] query - laziness of lateral join with function

2015-02-12 Thread Tom Lane
paulcc writes: >select count(alpha.id) >from alpha >cross join lateral some_function(alpha.id) as some_val >where alpha.test >Here the function is strict, and moreover its argument will never >be null - hence there should always be a non-null value returned. >I woul

[PERFORM] query - laziness of lateral join with function

2015-02-12 Thread paulcc
Hi I'm using cross join lateral with a non-trivial function in an attempt to limit calculation of that function, and am wondering about some aspects of how lateral is currently implemented. NB these queries are generated by a certain ORM, and are usually embedded in much more complex queries.

Re: [PERFORM] Survey: Max TPS you've ever seen

2015-02-12 Thread Graeme B. Bell
>> 1. O/S Under "O/S", don't forget to mention linux kernel version. We saw a MASSIVE increase in TPS (I think it was a doubling? Don't have the data to hand right now) on our multicore RHEL6 servers, when moving from a stock RHEL6 kernel to an ELREPO 3.18 series kernel. That's what 10 years

Re: [PERFORM] Survey: Max TPS you've ever seen

2015-02-12 Thread Gudmundsson Martin (mg)
Hi all! > - checkpoint_segments 1000 > - checkpoint_completion_target 0.9 > - wal_buffers  256MB > - shared_buffers 31 gb > - max_connections 500 I see that some of you are using wal_buffers = 256MB. I was under the impression that Postgres will not benefit from higher value than the segment siz

Re: [PERFORM] Survey: Max TPS you've ever seen

2015-02-12 Thread Luis Antonio Dias de Sá Junior
For me 12000 tps until now 24 core, 150 Gb ram - 5 ssd raid 5 - Debian 7.8 - Postgres 9.3.5 ...with Postgres parameters customized: - checkpoint_segments 1000 - checkpoint_completion_target 0.9 - wal_buffers 256MB - shared_buffers 31 gb - max_connections 500 - effective_io_concurrency 15 ..an