Re: [PERFORM] hash join vs nested loop join

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 8:57 AM, Evgeny Shishkin wrote: > > On Dec 12, 2012, at 8:44 AM, Huan Ruan wrote: > >> >> On 12 December 2012 15:33, Evgeny Shishkin wrote: >> Optimiser thinks that nested loop is more expensive, because of point PK >> lookups, which a random io. >> Can you set random_p

Re: [PERFORM] hash join vs nested loop join

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 8:25 AM, Huan Ruan wrote: > Hello All > > While investigating switching to Postgres, we come across a query plan that > uses hash join and is a lot slower than a nested loop join. > > I don't understand why the optimiser chooses the hash join in favor of the > nested loop

[PERFORM] hash join vs nested loop join

2012-12-11 Thread Huan Ruan
Hello All While investigating switching to Postgres, we come across a query plan that uses hash join and is a lot slower than a nested loop join. I don't understand why the optimiser chooses the hash join in favor of the nested loop. What can I do to get the optimiser to make a better decision (n

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread David Boreham
On 12/11/2012 8:11 PM, Evgeny Shishkin wrote: Quoting http://www.storagereview.com/intel_ssd_dc_s3700_series_enterprise_ssd_review Heh. A fine example of the kind of hand-waving of which I spoke ;) Higher performance is certainly a benefit, although at present we can't saturate even a single

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 7:05 AM, David Boreham wrote: > On 12/11/2012 7:49 PM, Evgeny Shishkin wrote: >> Yeah, s3700 looks promising, but sata interface is limiting factor for this >> drive. >> I'm looking towards SMART ssd >> http://www.storagereview.com/smart_storage_systems_optimus_sas_enterpri

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread David Boreham
On 12/11/2012 7:49 PM, Evgeny Shishkin wrote: Yeah, s3700 looks promising, but sata interface is limiting factor for this drive. I'm looking towards SMART ssd http://www.storagereview.com/smart_storage_systems_optimus_sas_enterprise_ssd_review What don't you like about SATA ? I prefer to avo

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Mark Kirkwood
On 12/12/12 15:41, David Boreham wrote: On 12/11/2012 7:38 PM, Evgeny Shishkin wrote: Which drives would you recommend? Besides intel 320 and 710. Those are the only drive types we have deployed in servers at present (almost all 710, but we have some 320 for less mission-critical machines). Th

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 6:41 AM, David Boreham wrote: > On 12/11/2012 7:38 PM, Evgeny Shishkin wrote: >> Which drives would you recommend? Besides intel 320 and 710. > Those are the only drive types we have deployed in servers at present (almost > all 710, but we have some 320 for less mission-crit

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Craig Ringer
On 12/12/2012 10:13 AM, Evgeny Shishkin wrote: > > Yes, i am aware of this issue. Never experienced this neither on intel > 520, no ocz vertex 3. > I wouldn't trust either of those drives. The 520 doesn't have Intel's " Enhanced Power Loss Data Protection"; it's going to lose its buffers if it los

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread David Boreham
On 12/11/2012 7:38 PM, Evgeny Shishkin wrote: Which drives would you recommend? Besides intel 320 and 710. Those are the only drive types we have deployed in servers at present (almost all 710, but we have some 320 for less mission-critical machines). The new DC-S3700 Series looks nice too, but

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 6:26 AM, David Boreham wrote: > On 12/11/2012 7:20 PM, Evgeny Shishkin wrote: >> Oh, there is no 100% safe system. > In this case we're discussing specifically "safety in the event of power loss > shortly after the drive indicates to the controller that it has committed a >

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread David Boreham
On 12/11/2012 7:20 PM, Evgeny Shishkin wrote: Oh, there is no 100% safe system. In this case we're discussing specifically "safety in the event of power loss shortly after the drive indicates to the controller that it has committed a write operation". Some drives do provide 100% safety against

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 6:15 AM, David Boreham wrote: > On 12/11/2012 7:13 PM, Evgeny Shishkin wrote: >> Yes, i am aware of this issue. Never experienced this neither on intel 520, >> no ocz vertex 3. >> Have you heard of them on this list? > People have done plug-pull tests and reported the result

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread David Boreham
On 12/11/2012 7:13 PM, Evgeny Shishkin wrote: Yes, i am aware of this issue. Never experienced this neither on intel 520, no ocz vertex 3. Have you heard of them on this list? People have done plug-pull tests and reported the results on the list (sometime in the past couple of years). But you

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 6:02 AM, Craig Ringer wrote: > On 12/12/2012 09:44 AM, Evgeny Shishkin wrote: >> So far, more than a year already, i bought consumer ssds with 300-400$ hw >> raid. Cost effective and fast, may be not very safe, but so far so good. All >> data protection measures from postgr

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Craig Ringer
On 12/12/2012 09:44 AM, Evgeny Shishkin wrote: > So far, more than a year already, i bought consumer ssds with 300-400$ > hw raid. Cost effective and fast, may be not very safe, but so far so > good. All data protection measures from postgresql are on, of course. You're aware that many low end SSD

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Rosser Schwarz
On Tue, Dec 11, 2012 at 5:17 PM, Evgeny Shishkin wrote: > Actually most of low-end SSDs don't do write caching, they do not have > enough ram for that. > AIUI, *all* SSDs do write-caching of a sort: writes are actually flushed to the NAND media by erasing, and then overwriting the erased space, a

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 5:29 AM, Craig Ringer wrote: > On 12/12/2012 09:17 AM, Evgeny Shishkin wrote: >> >> Actually most of low-end SSDs don't do write caching, they do not have >> enough ram for that. Sandforce for example. >> > Or, worse, some of them do limited write caching but don't protect

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Craig Ringer
On 12/12/2012 09:17 AM, Evgeny Shishkin wrote: > > Actually most of low-end SSDs don't do write caching, they do not have > enough ram for that. Sandforce for example. > Or, worse, some of them do limited write caching but don't protect their write cache from power loss. Instant data corruption! I

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 5:03 AM, Craig Ringer wrote: > On 12/12/2012 06:44 AM, Evgeny Shishkin wrote: >> >> On Dec 12, 2012, at 2:41 AM, Niels Kristian Schjødt >> wrote: >> >>> Are you using a hardware based raid controller with them? >>> >> Yes, of course. Hardware raid with cache and bbu is a

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Craig Ringer
On 12/12/2012 06:44 AM, Evgeny Shishkin wrote: > > On Dec 12, 2012, at 2:41 AM, Niels Kristian Schjødt > mailto:nielskrist...@autouncle.com>> wrote: > >> Are you using a hardware based raid controller with them? >> > Yes, of course. Hardware raid with cache and bbu is a must. You can't > get fast f

Re: [PERFORM] Why is PostgreSQL 9.2 slower than 9.1 in my tests?

2012-12-11 Thread Tom Lane
Jeff Janes writes: > On Tue, Dec 11, 2012 at 2:50 AM, Patryk Sidzina > wrote: >> The differences come up when you change the "INSERT" to "EXECUTE 'INSERT'" ( >> and i checked this time on 3 machines, one of which was Windows): >> FOR i IN 1..cnt LOOP >> EXECUTE 'INSERT INTO test_table_md_speed(n

Re: [PERFORM] Why is PostgreSQL 9.2 slower than 9.1 in my tests?

2012-12-11 Thread Jeff Janes
On Tue, Dec 11, 2012 at 2:50 AM, Patryk Sidzina wrote: > The differences come up when you change the "INSERT" to "EXECUTE 'INSERT'" ( > and i checked this time on 3 machines, one of which was Windows): > > CREATE TEMP TABLE test_table_md_speed(id serial primary key, n integer); > > CREATE OR REPL

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 12, 2012, at 2:41 AM, Niels Kristian Schjødt wrote: > Are you using a hardware based raid controller with them? > Yes, of course. Hardware raid with cache and bbu is a must. You can't get fast fsync without it. Also mdadm is a pain in the ass and is suitable only on amazon and other cl

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Niels Kristian Schjødt
Are you using a hardware based raid controller with them? Den 11/12/2012 20.11 skrev "Evgeny Shishkin" : > > On Dec 11, 2012, at 10:54 PM, Niels Kristian Schjødt < > nielskrist...@autouncle.com> wrote: > > And what is your experience so far? > > Increased tps by a factor of 10, database no longer

Re: [PERFORM] Occasional timeouts on TRUNCATE and simple INSERTs

2012-12-11 Thread Osborn, Jeff
Yeah I've been running a cron pulling relevant info from pg_stat_activity. Haven't seen anything yet. Currently looking into the pg_dump situation. --Jeff O On Dec 11, 2012, at 5:16 PM, Merlin Moncure wrote: > On Tue, Dec 11, 2012 at 3:38 PM, Sergey Konoplev wrote: > > Yeah: absolute first

Re: [PERFORM] Occasional timeouts on TRUNCATE and simple INSERTs

2012-12-11 Thread Merlin Moncure
On Tue, Dec 11, 2012 at 3:38 PM, Sergey Konoplev wrote: > On Tue, Dec 11, 2012 at 1:19 PM, Osborn, Jeff wrote: >> I am seeing some strange performance on a new pg9.1 instance. We are seeing >> occasional statement timeouts on some TRUNCATEs and INSERTs. In both cases, >> the statements are qu

Re: [PERFORM] Occasional timeouts on TRUNCATE and simple INSERTs

2012-12-11 Thread Sergey Konoplev
On Tue, Dec 11, 2012 at 1:19 PM, Osborn, Jeff wrote: > I am seeing some strange performance on a new pg9.1 instance. We are seeing > occasional statement timeouts on some TRUNCATEs and INSERTs. In both cases, > the statements are quite simple: >- TRUNCATE schema.table; > - INSERT INTO

[PERFORM] Occasional timeouts on TRUNCATE and simple INSERTs

2012-12-11 Thread Osborn, Jeff
Hello, I am seeing some strange performance on a new pg9.1 instance. We are seeing occasional statement timeouts on some TRUNCATEs and INSERTs. In both cases, the statements are quite simple: - TRUNCATE schema.table; - INSERT INTO schema.table VALUES ($1,2,$2,''); Sometimes these will

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 11, 2012, at 10:54 PM, Niels Kristian Schjødt wrote: > And what is your experience so far? > Increased tps by a factor of 10, database no longer a limiting factor of application. And it is cheaper than brand rotating drives. > Den 11/12/2012 18.16 skrev "Evgeny Shishkin" : > > On De

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Niels Kristian Schjødt
And what is your experience so far? Den 11/12/2012 18.16 skrev "Evgeny Shishkin" : > > On Dec 11, 2012, at 5:35 PM, Niels Kristian Schjødt < > nielskrist...@autouncle.com> wrote: > > > > > Den 11/12/2012 kl. 14.29 skrev Craig Ringer : > > > >> On 12/11/2012 06:04 PM, Niels Kristian Schjødt wrote:

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Jeff Janes
On Tue, Dec 11, 2012 at 2:04 AM, Niels Kristian Schjødt wrote: > Den 11/12/2012 kl. 00.58 skrev Jeff Janes : > >> >> The fact that there is much more writing than reading tells me that >> most of your indexes are in RAM. The amount of index you are rapidly >> reading and dirtying is large enough

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Evgeny Shishkin
On Dec 11, 2012, at 5:35 PM, Niels Kristian Schjødt wrote: > > Den 11/12/2012 kl. 14.29 skrev Craig Ringer : > >> On 12/11/2012 06:04 PM, Niels Kristian Schjødt wrote: >>> >>> Maybe I should mention, that I never see more than max 5Gb out of my total >>> 32Gb being in use on the server… Can

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Niels Kristian Schjødt
Den 11/12/2012 kl. 14.29 skrev Craig Ringer : > On 12/11/2012 06:04 PM, Niels Kristian Schjødt wrote: >> >> Maybe I should mention, that I never see more than max 5Gb out of my total >> 32Gb being in use on the server… Can I somehow utilize more of it? > For an update-mostly workload it probabl

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Craig Ringer
On 12/11/2012 06:04 PM, Niels Kristian Schjødt wrote: > > Maybe I should mention, that I never see more than max 5Gb out of my total > 32Gb being in use on the server… Can I somehow utilize more of it? For an update-mostly workload it probably won't do you tons of good so long as all your indexes

Re: [PERFORM] Do I have a hardware or a software problem?

2012-12-11 Thread Niels Kristian Schjødt
Den 11/12/2012 kl. 00.58 skrev Jeff Janes : > On Mon, Dec 10, 2012 at 2:51 PM, Niels Kristian Schjødt > wrote: > >> synchronous_commit = off >> >> The pg_xlog folder has been moved onto the SSD array (md3), and symlinked >> back into the postgres dir. > > With synchronous_commit = off, or with