[PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
Hello, I just upgraded with pg_dump/restore from PostgreSQL 8.3.11 to 8.4.4 but I'm having major performance problems with a query with many left joins. Problem is that costs are now very, very, very high (was ok in 8.3). Analyze has been done. Indexes are of course there. - Merge Left

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Scott Marlowe
On Mon, Aug 30, 2010 at 12:20 AM, Gerhard Wiesinger li...@wiesinger.com wrote: Hello, I just upgraded with pg_dump/restore from PostgreSQL 8.3.11 to 8.4.4 but I'm having major performance problems with a query with many left joins. Problem is that costs are now very, very, very high (was ok

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Andreas Kretschmer
Gerhard Wiesinger li...@wiesinger.com wrote: I know that the data model is key/value pairs but it worked well in 8.3. I need this flexibility. Any ideas? If i understand the query correctly it's a pivot-table, right? If yes, and if i where you, i would try to rewrite this query, to

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Scott Marlowe wrote: On Mon, Aug 30, 2010 at 12:20 AM, Gerhard Wiesinger li...@wiesinger.com wrote: Hello, I just upgraded with pg_dump/restore from PostgreSQL 8.3.11 to 8.4.4 but I'm having major performance problems with a query with many left joins. Problem is that

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Scott Marlowe
On Mon, Aug 30, 2010 at 1:25 AM, Gerhard Wiesinger li...@wiesinger.com wrote: On Mon, 30 Aug 2010, Scott Marlowe wrote: On Mon, Aug 30, 2010 at 12:20 AM, Gerhard Wiesinger li...@wiesinger.com wrote: Hello, I just upgraded with pg_dump/restore from PostgreSQL 8.3.11 to 8.4.4 but I'm

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Pavel Stehule
Hello 2010/8/30 Andreas Kretschmer akretsch...@spamfence.net: Gerhard Wiesinger li...@wiesinger.com wrote: I know that the data model is key/value pairs but it worked well in 8.3. I need this flexibility. Any ideas? If i understand the query correctly it's a pivot-table, right? no -

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Andreas Kretschmer wrote: Gerhard Wiesinger li...@wiesinger.com wrote: I know that the data model is key/value pairs but it worked well in 8.3. I need this flexibility. Any ideas? If i understand the query correctly it's a pivot-table, right? The view flattens the

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Yeb Havinga
Jose Ildefonso Camargo Tolosa wrote: Also, nowadays, Intel has better performance than AMD, at least when comparing Athlon 64 vs Core2, I'm still saving to get a Phenom II system in order to benchmark them and see how it goes (does anyone have one of these for testing?).

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Scott Marlowe wrote: On Mon, Aug 30, 2010 at 1:25 AM, Gerhard Wiesinger li...@wiesinger.com wrote: On Mon, 30 Aug 2010, Scott Marlowe wrote: On Mon, Aug 30, 2010 at 12:20 AM, Gerhard Wiesinger li...@wiesinger.com wrote: Hello, I just upgraded with pg_dump/restore from

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Pavel Stehule wrote: Hello 2010/8/30 Andreas Kretschmer akretsch...@spamfence.net: Gerhard Wiesinger li...@wiesinger.com wrote: I know that the data model is key/value pairs but it worked well in 8.3. I need this flexibility. Any ideas? If i understand the query

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Pavel Stehule
2010/8/30 Gerhard Wiesinger li...@wiesinger.com: On Mon, 30 Aug 2010, Pavel Stehule wrote: Hello 2010/8/30 Andreas Kretschmer akretsch...@spamfence.net: Gerhard Wiesinger li...@wiesinger.com wrote: I know that the data model is key/value pairs but it worked well in 8.3. I need this

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Scott Carey
On Aug 27, 2010, at 10:25 AM, Greg Smith wrote: Scott Carey wrote: But the select count(*) query, cached in RAM is 3x faster in one system than the other. The CPUs aren't 3x different performance wise. Something else may be wrong here. An individual Core2 Duo 2.93Ghz should be at most

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Scott Marlowe
On Mon, Aug 30, 2010 at 1:58 AM, Yeb Havinga yebhavi...@gmail.com wrote: four parallel r...@p:~/ff/www.cs.virginia.edu/stream/FTP/Code# ./a.out ./a.out ./a.out ./a.out You know you can just do stream 4 to get 4 parallel streams right? -- Sent via pgsql-performance mailing list

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Yeb Havinga
Scott Marlowe wrote: On Mon, Aug 30, 2010 at 1:58 AM, Yeb Havinga yebhavi...@gmail.com wrote: four parallel r...@p:~/ff/www.cs.virginia.edu/stream/FTP/Code# ./a.out ./a.out ./a.out ./a.out You know you can just do stream 4 to get 4 parallel streams right? Which version is that?

[PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Eliot Gable
Not sure if anyone else saw this, but it struck me as an interesting idea if it could be added to PostgreSQL. GPU accelerated database operations could be very... interesting. Of course, this could be difficult to do in a way that usefully increases performance of PostgreSQL, but I'll leave that

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Greg Smith
Eliot Gable wrote: Not sure if anyone else saw this, but it struck me as an interesting idea if it could be added to PostgreSQL. GPU accelerated database operations could be very... interesting. Of course, this could be difficult to do in a way that usefully increases performance of PostgreSQL,

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Greg Smith
Scott Carey wrote: The 2427 should do 12.8 GB/sec theoretical peak (dual channel 800Mhz DDR2) per processor socket (so 2x that if multithreaded and 2 Sockets). A Nehalem will do ~2x that (triple channel, 1066Mhz) and is also significantly faster clock for clock. But a Core2 based Xeon on

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Greg Smith
Yeb Havinga wrote: model name : AMD Phenom(tm) II X4 940 Processor @ 3.00GHz cpu cores : 4 stream compiled with -O3 Function Rate (MB/s) Avg time Min time Max time Triad: 5395.1815 0.0089 0.0089 0.0089 For comparison sake, an only moderately

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Gaël Le Mignot
Hello, In my humble opinion, while it can sound interesting from a theorical point of view to outloads some operations to the GPU, there is a huge pratical problem in current world : databases which are big enough to require such heavy optimization are usually runned on server hardware, which

[PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Eliot Gable
Not sure if anyone else saw this, but it struck me as an interesting idea if it could be added to PostgreSQL. GPU accelerated database operations could be very... interesting. Of course, this could be difficult to do in a way that usefully increases performance of PostgreSQL, but I'll leave that

Re: [PERFORM] [Fwd: postgres 8.4.1 number of connections]

2010-08-30 Thread Maria L. Wilson
thanks for your response we have 3 app servers that attach to this one particular database server. What kind of load issues did you find on 8.4.1? I'd be interested in anything documented on it - this might make an upgrade a higher priority! thanks, Maria Scott Marlowe wrote: On Thu,

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Jose Ildefonso Camargo Tolosa
Hi! Thanks you all for this great amount of information! What memory/motherboard (ie, chipset) is installed on the phenom ii one? it looks like it peaks to ~6.2GB/s with 4 threads. Also, what kernel is on it? (uname -a would be nice). Now, this looks like sustained memory speed, what about

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Pavel Stehule wrote: 2010/8/30 Gerhard Wiesinger li...@wiesinger.com: On Mon, 30 Aug 2010, Pavel Stehule wrote: Hello 2010/8/30 Andreas Kretschmer akretsch...@spamfence.net: Gerhard Wiesinger li...@wiesinger.com wrote: I know that the data model is key/value pairs

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Tom Lane
Gerhard Wiesinger li...@wiesinger.com writes: I know the drawbacks of an EAV design but I don't want to discuss that. I want to discuss the major performance decrease of PostgreSQL 8.3 (performance was ok) to PostgreSQL 8.4 (performance is NOT ok). Any further ideas how I can track this

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Gerhard Wiesinger
On Mon, 30 Aug 2010, Tom Lane wrote: Gerhard Wiesinger li...@wiesinger.com writes: I know the drawbacks of an EAV design but I don't want to discuss that. I want to discuss the major performance decrease of PostgreSQL 8.3 (performance was ok) to PostgreSQL 8.4 (performance is NOT ok). Any

Re: [PERFORM] Major performance problem after upgrade from 8.3 to 8.4

2010-08-30 Thread Tom Lane
Gerhard Wiesinger li...@wiesinger.com writes: BTW: I have the old data setup. /var/lib/pgsql-old. Is there a fast setup with old version on different TCP port possible to compare query plans? You'll need to reinstall the old executables. If you put the new executables in the same directories,

Re: [PERFORM] Using Between

2010-08-30 Thread Ozer, Pam
Yes. ANALYZE was run after we loaded the data. Thanks for your assistance Here is the full Query. select distinct VehicleUsed.VehicleUsedId as VehicleUsedId , VehicleUsed.VehicleUsedDisplayPriority as VehicleUsedDisplayPriority , VehicleUsed.HasVehicleUsedThumbnail as HasVehicleUsedThumbnail

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Eliot Gable
Well, from that perspective, it becomes a chicken and egg problem. Without the software support to use a GPU in a server for acceleration, nobody's going to build a server with a GPU. However, as previously stated, I can understand the challenges with determining whether the offloading would even

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Yeb Havinga
Greg Smith wrote: This comes up every year or so. The ability of GPU offloading to help with sorting has to overcome the additional latency that comes from copying everything over to it and then getting all the results back. If you look at the typical types of sorting people see in

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Clemens Eisserer
Hi, This isn't an older Opteron, its 6 core, 6MB L3 cache Istanbul.  Its not the newer stuff either. Everything before Magny Cours is now an older Opteron from my perspective. The 6-cores are identical to Magny Cours (except that Magny Cours has two of those beast in one package). - Clemens

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Scott Marlowe
On Mon, Aug 30, 2010 at 8:56 AM, Gaël Le Mignot g...@pilotsystems.net wrote: Hello, In my humble opinion, while  it can sound interesting from a theorical point of view to outloads some  operations to the GPU, there is a huge pratical problem in current world  : databases which are big enough

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread David Boreham
Feels like I fell through a worm hole in space/time, back to inmos in 1987, and a guy from marketing has just walked in the office going on about there's a customer who wants to use our massively parallel hardware to speed up databases... -- Sent via pgsql-performance mailing list

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Jeff Davis
On Mon, 2010-08-30 at 09:51 -0400, Eliot Gable wrote: Not sure if anyone else saw this, but it struck me as an interesting idea if it could be added to PostgreSQL. GPU accelerated database operations could be very... interesting. Of course, this could be difficult to do in a way that usefully

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Chris Browne
david_l...@boreham.org (David Boreham) writes: Feels like I fell through a worm hole in space/time, back to inmos in 1987, and a guy from marketing has just walked in the office going on about there's a customer who wants to use our massively parallel hardware to speed up databases... ... As

[PERFORM] slow DDL creation

2010-08-30 Thread Kevin Kempter
Hi all ; we have an automated partition creation process that includes the creation of an FK constraint. we have a few other servers with similar scenarios and this is the only server that stinks per when we create the new partitions. Anyone have any thoughts on how to debug this? were

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread David Boreham
On 8/30/2010 3:18 PM, Chris Browne wrote: ... As long as you're willing to rewrite PostgreSQL in Occam 2... Just re-write it in Google's new language 'Go' : it's close enough to Occam and they'd probably fund the project.. ;) -- Sent via pgsql-performance mailing list

Re: [PERFORM] slow DDL creation

2010-08-30 Thread bricklen
On Mon, Aug 30, 2010 at 3:28 PM, Kevin Kempter cs_...@consistentstate.com wrote: Hi all ; we have an automated partition creation process that includes the creation of an FK constraint. we have a few other servers with similar scenarios and this is the only server that stinks per when we

Re: [PERFORM] slow DDL creation

2010-08-30 Thread David Fetter
On Mon, Aug 30, 2010 at 04:28:25PM -0600, Kevin Kempter wrote: Hi all ; we have an automated partition creation process that includes the creation of an FK constraint. we have a few other servers with similar scenarios and this is the only server that stinks per when we create the new

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Scott Carey
On a similar note, is Postgres' Quicksort a dual-pivot quicksort? This can be up to 2x as fast as a normal quicksort (25% fewer swap operations, and swap operations are more expensive than compares for most sorts). Just google 'dual pivot quicksort' for more info. And before anyone asks --

Re: [PERFORM] Performance on new 64bit server compared to my 32bit desktop

2010-08-30 Thread Jose Ildefonso Camargo Tolosa
Hi! Thanks for the review link! Ildefonso. On Mon, Aug 30, 2010 at 6:01 PM, Greg Smith g...@2ndquadrant.com wrote: Clemens Eisserer wrote: Hi, This isn't an older Opteron, its 6 core, 6MB L3 cache Istanbul.  Its not the newer stuff either. Everything before Magny Cours is now an

Re: [PERFORM] GPU Accelerated Sorting

2010-08-30 Thread Tom Lane
Scott Carey sc...@richrelevance.com writes: On a similar note, is Postgres' Quicksort a dual-pivot quicksort? This can be up to 2x as fast as a normal quicksort (25% fewer swap operations, and swap operations are more expensive than compares for most sorts). In Postgres, the swaps are