Re: [PERFORM] Inserting 8MB bytea: just 25% of disk perf used?

2010-01-18 Thread Scott Carey
On Jan 18, 2010, at 3:20 AM, fka...@googlemail.com wrote: > Hello Pierre, > > thank You for these useful test commands. > > Here is what I did: > > > Pierre Frédéric Caillaud: > >> Try this : >> >> CREATE TABLE test AS SELECT * FROM yourtable; >> >> This will test write speed, an

Re: [PERFORM] Bad plan choice nestloop vs. hashjoin

2010-01-18 Thread Kenneth Marshall
On Mon, Jan 18, 2010 at 12:13:24PM -0500, Tom Lane wrote: > Kenneth Marshall writes: > > We have just upgraded our monitoring server software and > > now the following query for graphing the data performs > > abysmally with the default settings. Here is the results > > of the EXPLAIN ANALYZE run w

Re: [PERFORM] Bad plan choice nestloop vs. hashjoin

2010-01-18 Thread Tom Lane
Kenneth Marshall writes: > We have just upgraded our monitoring server software and > now the following query for graphing the data performs > abysmally with the default settings. Here is the results > of the EXPLAIN ANALYZE run with nestloops enabled: I poked at this a bit more and now think I s

Re: [PERFORM] Inserting 8MB bytea: just 25% of disk perf used?

2010-01-18 Thread fka...@googlemail.com
fka...@googlemail.com: > I'll try to execute these tests on a SSD > and/or Raid system. FYI: On a recent but mid-range performing SSD (128 MB size, serially writing 80-140 MB, 100 MB average) the situation was worse for some reason. No difference by fsync=on/off. Felix -- Sent via pgsql-per

[PERFORM] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-01-18 Thread Greg Stark
Looking at this patch for the commitfest I have a few questions. 1) You said you added an fsync of the new directory -- where is that I don't see it anywhere. 2) Why does the second pass to do the fsyncs read through fromdir to find all the filenames. I find that odd and counterintuitive. It woul

Re: [PERFORM] Inserting 8MB bytea: just 25% of disk perf used?

2010-01-18 Thread fka...@googlemail.com
Matthew Wakeling: > The data needs to be written first to the WAL, in order to provide > crash-safety. So you're actually writing 1600MB, not 800. I come back again to saving WAL to another disk. Now, after all, I wonder: Doesn't the server wait anyway until WAL was written to disk? So, if true

Re: [PERFORM] Inserting 8MB bytea: just 25% of disk perf used?

2010-01-18 Thread fka...@googlemail.com
Hannu Krosing: > did you also test this with fsync=off ? Yes. No significant difference. > I suspect that what you are seeing is the effect of randomly writing to > the index files. While sequential write performance can be up to > 80MB/sec on modern drives, sequential writes are an order of ma

Re: [PERFORM] Inserting 8MB bytea: just 25% of disk perf used?

2010-01-18 Thread fka...@googlemail.com
Dave Crooke: > If you don't need that level of consistency for your 8MB blobs, write them > to plain files named with some kind of id, and put the id in the database > instead of the blob. The problem here is that I later need to give access to the database via TCP to an external client. This cli

Re: [PERFORM] Inserting 8MB bytea: just 25% of disk perf used?

2010-01-18 Thread fka...@googlemail.com
Hannu Krosing: > On Thu, 2010-01-14 at 21:14 +0100, fka...@googlemail.com wrote: > > Thanks a lot for your reply. > > > > Hannu Krosing: > > > > > > 4 Core CPU 3 Ghz, WinXP, 1 TB SATA disk. > > > > > > try inserting the same data using 4 parallel connections or even 8 > > > parallel ones. > >

Re: [PERFORM] Inserting 8MB bytea: just 25% of disk perf used?

2010-01-18 Thread fka...@googlemail.com
Hello Pierre, thank You for these useful test commands. Here is what I did: Pierre Frédéric Caillaud: > Try this : > > CREATE TABLE test AS SELECT * FROM yourtable; > > This will test write speed, and TOAST compression speed. > Then try this: (1) Setting: * pg_xlog sym'l