Re: [HACKERS] CopyReadLineText optimization revisited

2011-02-22 Thread Robert Haas
On Fri, Feb 18, 2011 at 9:35 PM, Bruce Momjian br...@momjian.us wrote: Was this implemented?  Is it a TODO? It's not entirely clear to me that there's a meaningful win here. Speeding up COPY is already on the TODO list, with this link:

Re: [HACKERS] CopyReadLineText optimization revisited

2011-02-22 Thread Bruce Momjian
Robert Haas wrote: On Fri, Feb 18, 2011 at 9:35 PM, Bruce Momjian br...@momjian.us wrote: Was this implemented? ?Is it a TODO? It's not entirely clear to me that there's a meaningful win here. Speeding up COPY is already on the TODO list, with this link:

Re: [HACKERS] CopyReadLineText optimization revisited

2011-02-18 Thread Bruce Momjian
Was this implemented? Is it a TODO? --- Heikki Linnakangas wrote: I'm reviving the effort I started a while back to make COPY faster: http://archives.postgresql.org/pgsql-patches/2008-02/msg00100.php

Re: [HACKERS] CopyReadLineText optimization revisited

2010-08-27 Thread Dimitri Fontaine
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Ok. If we have to, we can keep that, it just requires more programming. After searching for a \n, we can peek at the previous byte to check if it's a backslash (and if it is, the one before that to see if it's a backslash too, and

[HACKERS] CopyReadLineText optimization revisited

2010-08-26 Thread Heikki Linnakangas
I'm reviving the effort I started a while back to make COPY faster: http://archives.postgresql.org/pgsql-patches/2008-02/msg00100.php http://archives.postgresql.org/pgsql-patches/2008-03/msg00015.php The patch I now have is based on using memchr() to search end-of-line. In a nutshell: * we

Re: [HACKERS] CopyReadLineText optimization revisited

2010-08-26 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: * we perform possible encoding conversion early, one input block at a time, rather than after splitting the input into lines. This allows us to assume in the later stages that the data is in server encoding, allowing us to

Re: [HACKERS] CopyReadLineText optimization revisited

2010-08-26 Thread Heikki Linnakangas
On 26/08/10 22:16, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: * instead of the byte-at-a-time loop in CopyReadLineText(), use memchr() to find the next NL/CR character. This is where the speedup comes from. That seems like the speedup, if any, would be

Re: [HACKERS] CopyReadLineText optimization revisited

2010-08-26 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 26/08/10 22:16, Tom Lane wrote: I think this is likely to break apps that have worked for years. I can't get excited about doing that in return for an 0-10% speedup that might only materialize on some platforms. If the numbers