Re: [PERFORM] insert performance for win32

2005-11-04 Thread Qingqing Zhou
""Magnus Hagander"" <[EMAIL PROTECTED]> wrote >> >> I'd like to use the win32 provided recv(), send() functions >> instead of redirect them to pgwin32_recv()/pgwin32_send(), >> just like libpq does. If we do this, we will lose some >> functionalities, but I'd like to see the performance >> differe

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Qingqing Zhou
""Merlin Moncure"" <[EMAIL PROTECTED]> wrote > ok, here is gprof output from newlines/no newlines > [newlines] > % cumulative self self total > time seconds secondscalls s/call s/call name > 19.03 0.67 0.671 0.67 3.20 MainLoop > 17.61

Re: [HACKERS] [PERFORM] insert performance for win32

2005-11-04 Thread Magnus Hagander
> >> AFAICS it is appropriate to move the sigsetjmp and > >> setup_cancel_handler calls in front of the per-line loop inside > >> MainLoop --- can anyone see a reason not to? > > > hm. mainloop is re-entrant, right? That means each \i > would reset the > > handler...what is downside to keepin

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Magnus Hagander
> > I'm inclined to treat this as an outright bug, not just a minor > certainly... > > > performance issue, because it implies that a sufficiently long psql > > script would probably crash a Windows machine. > > actually, it's worse than that, it's more of a dos on the > whole system, as window

Re: [HACKERS] [PERFORM] insert performance for win32

2005-11-04 Thread Bruce Momjian
David Fetter wrote: > On Fri, Nov 04, 2005 at 01:01:20PM -0500, Tom Lane wrote: > > "Merlin Moncure" <[EMAIL PROTECTED]> writes: > > > Nailed it. > > > > > problem is in mainloop.c -> setup_cancel_handler. Apparently you > > > can have multiple handlers and windows keeps track of them all, > > >

Re: [HACKERS] [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
David Fetter <[EMAIL PROTECTED]> writes: > On Fri, Nov 04, 2005 at 01:01:20PM -0500, Tom Lane wrote: >> I'm inclined to treat this as an outright bug, not just a minor >> performance issue, because it implies that a sufficiently long psql >> script would probably crash a Windows machine. > Ouch.

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Bruce Momjian
Tom Lane wrote: > "Merlin Moncure" <[EMAIL PROTECTED]> writes: > > Nailed it. > > > problem is in mainloop.c -> setup_cancel_handler. Apparently you can > > have multiple handlers and windows keeps track of them all, even if they > > do the same thing. Keeping track of so many system handles wou

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: >> AFAICS it is appropriate to move the sigsetjmp and >> setup_cancel_handler >> calls in front of the per-line loop inside MainLoop --- can anyone see >> a reason not to? > hm. mainloop is re-entrant, right? That means each \i would reset the > handle

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
> "Merlin Moncure" <[EMAIL PROTECTED]> writes: > > Nailed it. > > > problem is in mainloop.c -> setup_cancel_handler. Apparently you can > > have multiple handlers and windows keeps track of them all, even if they > > do the same thing. Keeping track of so many system handles would > > naturally

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: > Nailed it. > problem is in mainloop.c -> setup_cancel_handler. Apparently you can > have multiple handlers and windows keeps track of them all, even if they > do the same thing. Keeping track of so many system handles would > naturally slow the whol

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
Nailed it. problem is in mainloop.c -> setup_cancel_handler. Apparently you can have multiple handlers and windows keeps track of them all, even if they do the same thing. Keeping track of so many system handles would naturally slow the whole process down. Commenting that line times are flat as

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Tom Lane
Michal Taborsky <[EMAIL PROTECTED]> writes: > OK. Mystery (sort of) solved. After you told me it works for you I had > to assume the problem was somewhere else. And, indeed, it was, though > it's not too obvious. > The two attributes are actually not of tybe bigint, but of type > "crm_object_id

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: > ok, mingw gprof is claiming MainLoop is a culprit here, The only thing I can see that would be different for Windows is the SetConsoleCtrlHandler kernel call ... could that be expensive? Why do we have either sigsetjmp or setup_cancel_handler inside

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
ok, here is gprof output from newlines/no newlines [newlines] % cumulative self self total time seconds secondscalls s/call s/call name 19.03 0.67 0.671 0.67 3.20 MainLoop 17.61 1.29 0.62 500031 0.00

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Michal Taborsky
Tom Lane napsal(a): Michal Taborsky <[EMAIL PROTECTED]> writes: We are facing a performance problem with views consisting of several unioned tables. The simplified schema is as follows: Perhaps you should show us the real schema, because I cannot duplicate your complaint on the toy case you

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
> > "Merlin Moncure" <[EMAIL PROTECTED]> writes: > > yeah. I'm guessing problem is in the mingw flex/bison (which I really, > > really hope is not the case) or some other win32 specific block of code. > > I'm snooping around there... > > Maybe I'm confused here, but I thought we had established

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: > yeah. I'm guessing problem is in the mingw flex/bison (which I really, > really hope is not the case) or some other win32 specific block of code. > I'm snooping around there... Maybe I'm confused here, but I thought we had established that the local

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Tom Lane
Michal Taborsky <[EMAIL PROTECTED]> writes: > We are facing a performance problem with views consisting of several > unioned tables. The simplified schema is as follows: Perhaps you should show us the real schema, because I cannot duplicate your complaint on the toy case you show. regression=# e

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
> That's bizarre ... I'd have thought a very long line would be more > likely to trigger internal performance problems than the original. > > What happens if you read the file with "psql -f dump.sql" instead > of cat/stdin? non-flat. Also ran via \i and got non flat times. > BTW, I get flat tim

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: > It doesn't, and it doesn't. :/ Ok, here's where it gets interesting. I > removed all the newlines from the test output (dump.sql) and got flat > times ;). That's bizarre ... I'd have thought a very long line would be more likely to trigger internal

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Michael Fuhr
On Fri, Nov 04, 2005 at 12:38:30PM +0100, Michal Taborsky wrote: > SELECT object_id FROM commonview WHERE link_id=1234567 > > The result is sequential scan on all tables, append, sort and then > filter scan on this whole thing. Which of course is slow as hell. We use > version 8.0.2. I couldn't

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
> > not quite: attached is a file to generate test. > > > cat dump.sql | psql -q yadda > > Ah. Does your psql have readline support? if so, does adding -n to > that command change anything? > It doesn't, and it doesn't. :/ Ok, here's where it gets interesting. I removed all the newlines fr

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Merlin Moncure
> Hello everyone. > > We are facing a performance problem with views consisting of several > unioned tables. The simplified schema is as follows: > > CREATE TABLE foo ( > foo_object_id bigint, > link_id bigint, > somedatatext, > PRIMARY KEY (foo_object_id

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Richard Huxton
Michal Taborsky wrote: ... UNION ... The result is sequential scan on all tables, append, sort and then filter scan on this whole thing. Which of course is slow as hell. We use version 8.0.2. And now the question: Is there a way to force the planner to push the condition lower, so it will u

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: >> You mean something like the attached? > not quite: attached is a file to generate test. > cat dump.sql | psql -q yadda Ah. Does your psql have readline support? if so, does adding -n to that command change anything? reg

[PERFORM] Searching union views not using indices

2005-11-04 Thread Michal Taborsky
Hello everyone. We are facing a performance problem with views consisting of several unioned tables. The simplified schema is as follows: CREATE TABLE foo ( foo_object_id bigint, link_id bigint, somedatatext, PRIMARY KEY (foo_object_id) ); CRE

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
> > You mean something like the attached? oh, btw I ran timeit.c and performance is flat and fairly fast. I'm pretty sure psql is the culprit here. Merlin ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
> You mean something like the attached? not quite: attached is a file to generate test. to do it: psql yadda \i timeit.sql \t \o dump.sql select make_dump(5, false); \q cat dump.sql | psql -q yadda and see what pops out. I had to do it that way because redirecting psql to dump file caused