Re: [HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-06-08 Thread Pierre C
The linux kernel also uses it when it's availabe, see e.g. http://tomoyo.sourceforge.jp/cgi-bin/lxr/source/arch/x86/crypto/crc32c-intel.c If you guys are interested I have a Core i7 here, could run a little benchmark. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-06-07 Thread Pierre C
On Sunday 30 May 2010 18:29:31 Greg Stark wrote: On Sun, May 30, 2010 at 4:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: I read through that thread and couldn't find much discussion of alternative CRC implementations --- we spent all our time on arguing about whether we needed 64-bit CRC or not.

Re: [HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-06-07 Thread Andres Freund
On Monday 07 June 2010 12:37:13 Pierre C wrote: On Sunday 30 May 2010 18:29:31 Greg Stark wrote: On Sun, May 30, 2010 at 4:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: I read through that thread and couldn't find much discussion of alternative CRC implementations --- we spent all our time

Re: [HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-06-07 Thread Florian Pflug
On Jun 7, 2010, at 12:45 , Andres Freund wrote: On Monday 07 June 2010 12:37:13 Pierre C wrote: On Sunday 30 May 2010 18:29:31 Greg Stark wrote: On Sun, May 30, 2010 at 4:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: I read through that thread and couldn't find much discussion of alternative CRC

Re: [HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-06-07 Thread Yeb Havinga
Florian Pflug wrote: On Jun 7, 2010, at 12:45 , Andres Freund wrote: On Monday 07 June 2010 12:37:13 Pierre C wrote: On Sunday 30 May 2010 18:29:31 Greg Stark wrote: On Sun, May 30, 2010 at 4:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: I read through that thread and

Re: [HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-05-30 Thread Greg Stark
On Sun, May 30, 2010 at 5:29 PM, Greg Stark gsst...@mit.edu wrote: On Sun, May 30, 2010 at 4:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: I read through that thread and couldn't find much discussion of alternative CRC implementations --- we spent all our time on arguing about whether we needed

Re: [HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-05-30 Thread Andres Freund
On Sunday 30 May 2010 18:29:31 Greg Stark wrote: On Sun, May 30, 2010 at 4:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: I read through that thread and couldn't find much discussion of alternative CRC implementations --- we spent all our time on arguing about whether we needed 64-bit CRC or

Re: [HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-05-30 Thread Andres Freund
On Sunday 30 May 2010 18:43:12 Greg Stark wrote: On Sun, May 30, 2010 at 5:29 PM, Greg Stark gsst...@mit.edu wrote: On Sun, May 30, 2010 at 4:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: I read through that thread and couldn't find much discussion of alternative CRC implementations --- we

[HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-05-29 Thread Greg Stark
This sounds familiar. If you search back in the archives around 2004 or so I think you'll find a similar discussion when we replaced the crc32 implementation with what we have now. We put a fair amount of effort into searching for faster implementations so if you've found one 3x faster I'm pretty

[HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-05-29 Thread Greg Stark
On Sun, May 30, 2010 at 3:56 AM, Greg Stark gsst...@mit.edu wrote: This sounds familiar. If you search back in the archives around 2004 or so I think you'll find a similar discussion when we replaced the crc32 implementation with what we have now. Fwiw here's the thread (from 2005):

Re: [HACKERS] Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

2010-05-29 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: On Sun, May 30, 2010 at 3:56 AM, Greg Stark gsst...@mit.edu wrote: This sounds familiar. If you search back in the archives around 2004 or so I think you'll find a similar discussion when we replaced the crc32 implementation with what we have now. Fwiw