Re: [PATCHES] Libpq COPY optimization patch
"Alon Goldshuv" <[EMAIL PROTECTED]> writes: > Libpq copy speedup patch attached. No input buffer logic involved, just > removing the expensive PQconsumeInput call per putCopyData call, and leaving > parseInput as is, as discussed before. Applied (with a better comment) to HEAD and 8.1 branches. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] Libpq COPY optimization patch
Libpq copy speedup patch attached. No input buffer logic involved, just removing the expensive PQconsumeInput call per putCopyData call, and leaving parseInput as is, as discussed before. Alon. libpq_copy.patch Description: Binary data ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] Libpq COPY optimization patch
> You mean something like > > if (input-buffer-not-empty) > parseInput(); > > ? This still bothers me a bit since it's a mixing of logic levels; > PQputCopyData is an output routine, it shouldn't be getting its fingers > dirty with input buffer contents. I'm willing to tolerate this if it > can be demonstrated that it provides a useful performance gain compared > to the unconditional parseInput call, but let's see some numbers. Yes, I understand. We'll see what the performance gain is like and see if it's worth it, I'll report back. Alon. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] Libpq COPY optimization patch
"Alon Goldshuv" <[EMAIL PROTECTED]> writes: > I guess that although parseInput is cheap we could still use a conditional > to see when data was sent and only then call it (without PQconsumeInput) > instead of calling it every single time PQputCopyData is called. Any > objection to that? You mean something like if (input-buffer-not-empty) parseInput(); ? This still bothers me a bit since it's a mixing of logic levels; PQputCopyData is an output routine, it shouldn't be getting its fingers dirty with input buffer contents. I'm willing to tolerate this if it can be demonstrated that it provides a useful performance gain compared to the unconditional parseInput call, but let's see some numbers. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] Libpq COPY optimization patch
Tom, >> Here is a patch against today's code 1/24. As discussed in -hackers >> consumeInput/parse is removed from being called every single time. It's >> still there for only when the data is sent to the server. > > This appears to be the exact same patch you sent before. Did you > test my suggestion of simply removing the PQconsumeInput call? > I see no reason to add it inside the loop. My mistake. I'll make the correction. I guess that although parseInput is cheap we could still use a conditional to see when data was sent and only then call it (without PQconsumeInput) instead of calling it every single time PQputCopyData is called. Any objection to that? Alon. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Libpq COPY optimization patch
"Alon Goldshuv" <[EMAIL PROTECTED]> writes: > Here is a patch against today's code 1/24. As discussed in -hackers > consumeInput/parse is removed from being called every single time. It's > still there for only when the data is sent to the server. This appears to be the exact same patch you sent before. Did you test my suggestion of simply removing the PQconsumeInput call? I see no reason to add it inside the loop. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
[PATCHES] Libpq COPY optimization patch
Here is a patch against today's code 1/24. As discussed in -hackers consumeInput/parse is removed from being called every single time. It's still there for only when the data is sent to the server. Alon. pq_put_copy_data.patch Description: Binary data ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
