Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
You mean something like the attached? not quite: attached is a file to generate test. to do it: psql yadda \i timeit.sql \t \o dump.sql select make_dump(5, false); \q cat dump.sql | psql -q yadda and see what pops out. I had to do it that way because redirecting psql to dump file caused

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
You mean something like the attached? oh, btw I ran timeit.c and performance is flat and fairly fast. I'm pretty sure psql is the culprit here. Merlin ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

[PERFORM] Searching union views not using indices

2005-11-04 Thread Michal Taborsky
Hello everyone. We are facing a performance problem with views consisting of several unioned tables. The simplified schema is as follows: CREATE TABLE foo ( foo_object_id bigint, link_id bigint, somedatatext, PRIMARY KEY (foo_object_id) );

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
Merlin Moncure [EMAIL PROTECTED] writes: You mean something like the attached? not quite: attached is a file to generate test. cat dump.sql | psql -q yadda Ah. Does your psql have readline support? if so, does adding -n to that command change anything? regards,

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Richard Huxton
Michal Taborsky wrote: ... UNION ... The result is sequential scan on all tables, append, sort and then filter scan on this whole thing. Which of course is slow as hell. We use version 8.0.2. And now the question: Is there a way to force the planner to push the condition lower, so it will

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Merlin Moncure
Hello everyone. We are facing a performance problem with views consisting of several unioned tables. The simplified schema is as follows: CREATE TABLE foo ( foo_object_id bigint, link_id bigint, somedatatext, PRIMARY KEY (foo_object_id) );

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
not quite: attached is a file to generate test. cat dump.sql | psql -q yadda Ah. Does your psql have readline support? if so, does adding -n to that command change anything? It doesn't, and it doesn't. :/ Ok, here's where it gets interesting. I removed all the newlines from the

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Michael Fuhr
On Fri, Nov 04, 2005 at 12:38:30PM +0100, Michal Taborsky wrote: SELECT object_id FROM commonview WHERE link_id=1234567 The result is sequential scan on all tables, append, sort and then filter scan on this whole thing. Which of course is slow as hell. We use version 8.0.2. I couldn't

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
Merlin Moncure [EMAIL PROTECTED] writes: It doesn't, and it doesn't. :/ Ok, here's where it gets interesting. I removed all the newlines from the test output (dump.sql) and got flat times ;). That's bizarre ... I'd have thought a very long line would be more likely to trigger internal

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
That's bizarre ... I'd have thought a very long line would be more likely to trigger internal performance problems than the original. What happens if you read the file with psql -f dump.sql instead of cat/stdin? non-flat. Also ran via \i and got non flat times. BTW, I get flat times for

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Tom Lane
Michal Taborsky [EMAIL PROTECTED] writes: We are facing a performance problem with views consisting of several unioned tables. The simplified schema is as follows: Perhaps you should show us the real schema, because I cannot duplicate your complaint on the toy case you show. regression=#

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
Merlin Moncure [EMAIL PROTECTED] writes: yeah. I'm guessing problem is in the mingw flex/bison (which I really, really hope is not the case) or some other win32 specific block of code. I'm snooping around there... Maybe I'm confused here, but I thought we had established that the local and

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
Merlin Moncure [EMAIL PROTECTED] writes: yeah. I'm guessing problem is in the mingw flex/bison (which I really, really hope is not the case) or some other win32 specific block of code. I'm snooping around there... Maybe I'm confused here, but I thought we had established that the

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Michal Taborsky
Tom Lane napsal(a): Michal Taborsky [EMAIL PROTECTED] writes: We are facing a performance problem with views consisting of several unioned tables. The simplified schema is as follows: Perhaps you should show us the real schema, because I cannot duplicate your complaint on the toy case you

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
ok, here is gprof output from newlines/no newlines [newlines] % cumulative self self total time seconds secondscalls s/call s/call name 19.03 0.67 0.671 0.67 3.20 MainLoop 17.61 1.29 0.62 500031 0.00

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
Merlin Moncure [EMAIL PROTECTED] writes: ok, mingw gprof is claiming MainLoop is a culprit here, The only thing I can see that would be different for Windows is the SetConsoleCtrlHandler kernel call ... could that be expensive? Why do we have either sigsetjmp or setup_cancel_handler inside the

Re: [PERFORM] Searching union views not using indices

2005-11-04 Thread Tom Lane
Michal Taborsky [EMAIL PROTECTED] writes: OK. Mystery (sort of) solved. After you told me it works for you I had to assume the problem was somewhere else. And, indeed, it was, though it's not too obvious. The two attributes are actually not of tybe bigint, but of type crm_object_id, which

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
Nailed it. problem is in mainloop.c - setup_cancel_handler. Apparently you can have multiple handlers and windows keeps track of them all, even if they do the same thing. Keeping track of so many system handles would naturally slow the whole process down. Commenting that line times are flat as

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
Merlin Moncure [EMAIL PROTECTED] writes: Nailed it. problem is in mainloop.c - setup_cancel_handler. Apparently you can have multiple handlers and windows keeps track of them all, even if they do the same thing. Keeping track of so many system handles would naturally slow the whole process

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Merlin Moncure
Merlin Moncure [EMAIL PROTECTED] writes: Nailed it. problem is in mainloop.c - setup_cancel_handler. Apparently you can have multiple handlers and windows keeps track of them all, even if they do the same thing. Keeping track of so many system handles would naturally slow the whole

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
Merlin Moncure [EMAIL PROTECTED] writes: AFAICS it is appropriate to move the sigsetjmp and setup_cancel_handler calls in front of the per-line loop inside MainLoop --- can anyone see a reason not to? hm. mainloop is re-entrant, right? That means each \i would reset the handler...what is

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Bruce Momjian
Tom Lane wrote: Merlin Moncure [EMAIL PROTECTED] writes: Nailed it. problem is in mainloop.c - setup_cancel_handler. Apparently you can have multiple handlers and windows keeps track of them all, even if they do the same thing. Keeping track of so many system handles would naturally

Re: [HACKERS] [PERFORM] insert performance for win32

2005-11-04 Thread Tom Lane
David Fetter [EMAIL PROTECTED] writes: On Fri, Nov 04, 2005 at 01:01:20PM -0500, Tom Lane wrote: I'm inclined to treat this as an outright bug, not just a minor performance issue, because it implies that a sufficiently long psql script would probably crash a Windows machine. Ouch. In light

Re: [HACKERS] [PERFORM] insert performance for win32

2005-11-04 Thread Bruce Momjian
David Fetter wrote: On Fri, Nov 04, 2005 at 01:01:20PM -0500, Tom Lane wrote: Merlin Moncure [EMAIL PROTECTED] writes: Nailed it. problem is in mainloop.c - setup_cancel_handler. Apparently you can have multiple handlers and windows keeps track of them all, even if they do the

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Magnus Hagander
I'm inclined to treat this as an outright bug, not just a minor certainly... performance issue, because it implies that a sufficiently long psql script would probably crash a Windows machine. actually, it's worse than that, it's more of a dos on the whole system, as windows will

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Qingqing Zhou
Merlin Moncure [EMAIL PROTECTED] wrote ok, here is gprof output from newlines/no newlines [newlines] % cumulative self self total time seconds secondscalls s/call s/call name 19.03 0.67 0.671 0.67 3.20 MainLoop 17.61 1.29

Re: [HACKERS] [PERFORM] insert performance for win32

2005-11-04 Thread Magnus Hagander
AFAICS it is appropriate to move the sigsetjmp and setup_cancel_handler calls in front of the per-line loop inside MainLoop --- can anyone see a reason not to? hm. mainloop is re-entrant, right? That means each \i would reset the handler...what is downside to keeping global flag?

Re: [PERFORM] insert performance for win32

2005-11-04 Thread Qingqing Zhou
Magnus Hagander [EMAIL PROTECTED] wrote I'd like to use the win32 provided recv(), send() functions instead of redirect them to pgwin32_recv()/pgwin32_send(), just like libpq does. If we do this, we will lose some functionalities, but I'd like to see the performance difference first. -- do