Libpq copy speedup patch attached. No input buffer logic involved, just
removing the expensive PQconsumeInput call per putCopyData call, and leaving
parseInput as is, as discussed before.
Alon.
libpq_copy.patch
Description: Binary data
---(end of
Here is a patch against today's code 1/24. As discussed in -hackers
consumeInput/parse is removed from being called every single time. It's
still there for only when the data is sent to the server.
Alon.
pq_put_copy_data.patch
Description: Binary data
---(end of
Tom,
Here is a patch against today's code 1/24. As discussed in -hackers
consumeInput/parse is removed from being called every single time. It's
still there for only when the data is sent to the server.
This appears to be the exact same patch you sent before. Did you
test my suggestion of
You mean something like
if (input-buffer-not-empty)
parseInput();
? This still bothers me a bit since it's a mixing of logic levels;
PQputCopyData is an output routine, it shouldn't be getting its fingers
dirty with input buffer contents. I'm willing to tolerate this if it
can be
I did some performance checks after the recent code commit.
The good news is that the parsing speed of COPY is now MUCH faster, which is
great. It is about 5 times faster - about 100MB/sec on my machine
(previously 20MB/sec at best, usually less).
The better news is that my original patch
Tom,
Thanks for pointing it out. I made the small required modifications to match
copy.c version 1.247 and sent it to -patches list. New patch is V16.
Alon.
On 8/1/05 7:51 PM, Tom Lane [EMAIL PROTECTED] wrote:
Alon Goldshuv [EMAIL PROTECTED] writes:
This patch appears to reverse out
... This is for TEXT format. As for CSV, it may be faster but not as much
as I specified here. BINARY will stay the same as before.
HTH
Alon.
On 7/19/05 12:54 PM, Mark Wong [EMAIL PROTECTED] wrote:
On Thu, 14 Jul 2005 17:22:18 -0700
Alon Goldshuv [EMAIL PROTECTED] wrote:
I revisited my
nation.tbl
15000 orders.tbl
2000 part.tbl
8000 partsupp.tbl
5 region.tbl
100 supplier.tbl
Thanks,
Mark
On Tue, 19 Jul 2005 14:05:56 -0700
Alon Goldshuv [EMAIL PROTECTED] wrote:
Hi Mark,
I improved the data *parsing* capabilities of COPY
Neil,
Right; I really dislike the idea of having two separate code paths for
COPY. When you say this approach is temporary, are you suggesting that
you intend to reimplement your changes as improvements/replacements of
the existing COPY code path rather than as a parallel code path?
My
