On Fri, 2005-11-04 at 13:21 -0500, Bruce Momjian wrote:
David Fetter wrote:
On Fri, Nov 04, 2005 at 01:01:20PM -0500, Tom Lane wrote:
I'm inclined to treat this as an outright bug, not just a minor
performance issue, because it implies that a sufficiently long psql
script would
You mean something like the attached?
not quite: attached is a file to generate test.
to do it:
psql yadda
\i timeit.sql
\t
\o dump.sql
select make_dump(5, false);
\q
cat dump.sql | psql -q yadda
and see what pops out. I had to do it that way because redirecting psql
to dump file caused
You mean something like the attached?
oh, btw I ran timeit.c and performance is flat and fairly fast. I'm
pretty sure psql is the culprit here.
Merlin
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
Merlin Moncure [EMAIL PROTECTED] writes:
You mean something like the attached?
not quite: attached is a file to generate test.
cat dump.sql | psql -q yadda
Ah. Does your psql have readline support? if so, does adding -n to
that command change anything?
regards,
not quite: attached is a file to generate test.
cat dump.sql | psql -q yadda
Ah. Does your psql have readline support? if so, does adding -n to
that command change anything?
It doesn't, and it doesn't. :/ Ok, here's where it gets interesting. I
removed all the newlines from the
Merlin Moncure [EMAIL PROTECTED] writes:
It doesn't, and it doesn't. :/ Ok, here's where it gets interesting. I
removed all the newlines from the test output (dump.sql) and got flat
times ;).
That's bizarre ... I'd have thought a very long line would be more
likely to trigger internal
That's bizarre ... I'd have thought a very long line would be more
likely to trigger internal performance problems than the original.
What happens if you read the file with psql -f dump.sql instead
of cat/stdin?
non-flat. Also ran via \i and got non flat times.
BTW, I get flat times for
Merlin Moncure [EMAIL PROTECTED] writes:
yeah. I'm guessing problem is in the mingw flex/bison (which I really,
really hope is not the case) or some other win32 specific block of code.
I'm snooping around there...
Maybe I'm confused here, but I thought we had established that the local
and
Merlin Moncure [EMAIL PROTECTED] writes:
yeah. I'm guessing problem is in the mingw flex/bison (which I
really,
really hope is not the case) or some other win32 specific block of
code.
I'm snooping around there...
Maybe I'm confused here, but I thought we had established that the
ok, here is gprof output from newlines/no newlines
[newlines]
% cumulative self self total
time seconds secondscalls s/call s/call name
19.03 0.67 0.671 0.67 3.20 MainLoop
17.61 1.29 0.62 500031 0.00
Merlin Moncure [EMAIL PROTECTED] writes:
ok, mingw gprof is claiming MainLoop is a culprit here,
The only thing I can see that would be different for Windows is the
SetConsoleCtrlHandler kernel call ... could that be expensive? Why
do we have either sigsetjmp or setup_cancel_handler inside the
Nailed it.
problem is in mainloop.c - setup_cancel_handler. Apparently you can
have multiple handlers and windows keeps track of them all, even if they
do the same thing. Keeping track of so many system handles would
naturally slow the whole process down. Commenting that line times are
flat as
Merlin Moncure [EMAIL PROTECTED] writes:
Nailed it.
problem is in mainloop.c - setup_cancel_handler. Apparently you can
have multiple handlers and windows keeps track of them all, even if they
do the same thing. Keeping track of so many system handles would
naturally slow the whole process
Merlin Moncure [EMAIL PROTECTED] writes:
Nailed it.
problem is in mainloop.c - setup_cancel_handler. Apparently you
can
have multiple handlers and windows keeps track of them all, even if
they
do the same thing. Keeping track of so many system handles would
naturally slow the whole
Merlin Moncure [EMAIL PROTECTED] writes:
AFAICS it is appropriate to move the sigsetjmp and
setup_cancel_handler
calls in front of the per-line loop inside MainLoop --- can anyone see
a reason not to?
hm. mainloop is re-entrant, right? That means each \i would reset the
handler...what is
Tom Lane wrote:
Merlin Moncure [EMAIL PROTECTED] writes:
Nailed it.
problem is in mainloop.c - setup_cancel_handler. Apparently you can
have multiple handlers and windows keeps track of them all, even if they
do the same thing. Keeping track of so many system handles would
naturally
David Fetter [EMAIL PROTECTED] writes:
On Fri, Nov 04, 2005 at 01:01:20PM -0500, Tom Lane wrote:
I'm inclined to treat this as an outright bug, not just a minor
performance issue, because it implies that a sufficiently long psql
script would probably crash a Windows machine.
Ouch. In light
David Fetter wrote:
On Fri, Nov 04, 2005 at 01:01:20PM -0500, Tom Lane wrote:
Merlin Moncure [EMAIL PROTECTED] writes:
Nailed it.
problem is in mainloop.c - setup_cancel_handler. Apparently you
can have multiple handlers and windows keeps track of them all,
even if they do the
I'm inclined to treat this as an outright bug, not just a minor
certainly...
performance issue, because it implies that a sufficiently long psql
script would probably crash a Windows machine.
actually, it's worse than that, it's more of a dos on the
whole system, as windows will
Merlin Moncure [EMAIL PROTECTED] wrote
ok, here is gprof output from newlines/no newlines
[newlines]
% cumulative self self total
time seconds secondscalls s/call s/call name
19.03 0.67 0.671 0.67 3.20 MainLoop
17.61 1.29
AFAICS it is appropriate to move the sigsetjmp and
setup_cancel_handler calls in front of the per-line loop inside
MainLoop --- can anyone see a reason not to?
hm. mainloop is re-entrant, right? That means each \i
would reset the
handler...what is downside to keeping global flag?
Magnus Hagander [EMAIL PROTECTED] wrote
I'd like to use the win32 provided recv(), send() functions
instead of redirect them to pgwin32_recv()/pgwin32_send(),
just like libpq does. If we do this, we will lose some
functionalities, but I'd like to see the performance
difference first. -- do
On Wed, 2 Nov 2005, Merlin Moncure wrote:
If you put client/server on the same machine, then we don't know how
the
CPU is splitted. Can you take a look at the approximate number by
observing the task manager data while running?
ok, I generated a test case which was 250k inserts to simple two
Both win32 send/recv have pgwin32_poll_signals() in them. This is
glorified WaitForSingleObjectEx on global pgwin32_signal_event. This
is
probably part of the problem. Can we work some of the same magic you
put
into check interrupts macro?
Whoop! following a cvs update I see this is
On Thu, 3 Nov 2005, Magnus Hagander wrote:
Sorry, I don't follow you here - what do you mean to do? Remove the
event completely so we can't wait on it?
I'd like to use the win32 provided recv(), send() functions instead of
redirect them to pgwin32_recv()/pgwin32_send(), just like libpq
Both win32 send/recv have pgwin32_poll_signals() in them.
This is glorified WaitForSingleObjectEx on global
pgwin32_signal_event. This is probably part of the problem.
Can we work some of the same magic you put into check interrupts
macro?
Uh, we already do that, don't we?
On Thu, 3 Nov 2005, Magnus Hagander wrote:
Sorry, I don't follow you here - what do you mean to do? Remove the
event completely so we can't wait on it?
I'd like to use the win32 provided recv(), send() functions
instead of redirect them to pgwin32_recv()/pgwin32_send(),
just
I'd like to use the win32 provided recv(), send()
functions instead
of redirect them to pgwin32_recv()/pgwin32_send(), just
like libpq
does. If we do this, we will lose some functionalities,
but I'd like
to see the performance difference first. -- do you think
that will
Merlin Moncure [EMAIL PROTECTED] wrote
Running from remote, Time progression is:
First 50k: 20 sec
Second: 29 sec
[...]
final:: 66 sec
This may due to the maintainence cost of a big transaction, I am not sure
... Tom?
so, clear upward progression of time/rec. Initial time is
Qingqing Zhou [EMAIL PROTECTED] wrote
Not to 100%, so this means the server is always starving. It is waiting on
something -- of couse not lock. That's why I think there is some problem
on network communication. Another suspect will be the write - I knwo NTFS
system will issue an internal
On Thu, 3 Nov 2005, Magnus Hagander wrote:
Both win32 send/recv have pgwin32_poll_signals() in them.
This is glorified WaitForSingleObjectEx on global
pgwin32_signal_event. This is probably part of the problem.
Can we work some of the same magic you put into check
interrupts macro?
Merlin Moncure [EMAIL PROTECTED] writes:
ok, I generated a test case which was 250k inserts to simple two column
table all in single transaction. Every 50k inserts, time is recorded
via timeofday().
You mean something like the attached?
Running from remote, Time progression is:
First
On Thu, 3 Nov 2005, Tom Lane wrote:
On Unix I get a dead flat line (within measurement noise), both local
loopback and across my LAN.
after 5 30.20 sec
after 10 31.67 sec
after 15 30.98 sec
after 20 29.64 sec
after 25 29.83 sec
Confirmed in Linux. And on a winxp
I've done the tests with rc1. This is still as slow on windows ...
about
6-10
times slower thant linux (via Ip socket). (depending on using prepared
queries, etc...)
By the way, we've tried to insert into the windows database from a
linux
psql
client, via the network. In this
I've done the tests with rc1. This is still as slow on windows ...
about
6-10
times slower thant linux (via Ip socket). (depending on
using prepared
queries, etc...)
By the way, we've tried to insert into the windows database from a
linux
psql
client, via the network. In this
Le Mercredi 02 Novembre 2005 14:54, Magnus Hagander a écrit :
I've done the tests with rc1. This is still as slow on windows ...
about
6-10
times slower thant linux (via Ip socket). (depending on
using prepared
queries, etc...)
By the way, we've tried to insert into
Magnus Hagander [EMAIL PROTECTED] writes:
Marc's observation that by switching to a linux client drops
time down drastically is really intersing!
Could this be a case of the network being slow,
I'm wondering about nonstandard junk lurking in the TCP stack of the
Windows client machine.
On Wed, 2 Nov 2005, Merlin Moncure wrote:
By the way, we've tried to insert into the windows database from a
linux psql client, via the network. In this configuration, inserting
is only about 2 times slower than inserting locally (the linux client
had a slower CPU 1700Mhz agains
One thing I did notice that in a 250k insert transaction the insert
time
grows with #recs inserted. Time to insert first 50k recs is about
27
sec and last 50 k recs is 77 sec. I also confimed that size of
table is
not playing a role here.
Marc, can you do select timeofday() every 50k
On Tuesday 06 September 2005 19:11, Merlin Moncure wrote:
This makes me wonder if we are looking in the wrong place. Maybe the
problem is coming from psql? More results to follow.
problem is not coming from psql.
One thing I did notice that in a 250k insert transaction the insert time
On Tuesday 06 September 2005 19:11, Merlin Moncure wrote:
Here's the timeofday ... i'll do the gprof as soon as I can.
Every 5 rows...
Wed Sep 07 13:58:13.860378 2005 CEST
Wed Sep 07 13:58:20.926983 2005 CEST
Wed Sep 07 13:58:27.928385 2005 CEST
Wed Sep 07 13:58:35.472813 2005 CEST
Hi,
I usually use PostgreSQL coupled with Linux, but I have to use Windows for a
perticular project.
So I wanted to do some tests to know if the performance will be acceptable (I
don't need PostgreSQL to be as fast with windows as with linux, but it has to
be usable...).
I started with
Hi,
I usually use PostgreSQL coupled with Linux, but I have to use Windows
for
a
perticular project.
So I wanted to do some tests to know if the performance will be
acceptable
(I
don't need PostgreSQL to be as fast with windows as with linux, but it
has
to
be usable...).
In my
In my experience win32 is par with linux generally with a few gotchas on
either side. Are your times with fsync=no? It's much harder to give
apples-apples comparison with fsync=on for various reasons.
It is with fsync=off on windows, fsync=on on linux
Are you running
In my experience win32 is par with linux generally with a few gotchas on
either side. Are your times with fsync=no? It's much harder to give
apples-apples comparison with fsync=on for various reasons.
It is with fsync=off on windows, fsync=on on linux
well, inside a transaction this
This makes me wonder if we are looking in the wrong place. Maybe the
problem is coming from psql? More results to follow.
problem is not coming from psql.
One thing I did notice that in a 250k insert transaction the insert time
grows with #recs inserted. Time to insert first 50k recs is
46 matches
Mail list logo
