Re: [HACKERS] CUDA Sorting

2012-02-15 Thread Dann Corbit
-Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Gaetano Mendola Sent: Wednesday, February 15, 2012 2:54 PM To: Peter Geoghegan; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] CUDA Sorting On 15/02/2012 23:11, Peter

Re: [HACKERS] CUDA Sorting

2012-02-15 Thread Peter Geoghegan
On 15 February 2012 22:54, Gaetano Mendola wrote: > That sounds a bit harsh. I'm one of those indeed, I haven't look in the > details not having enough time for it. At work we do GPU computing (not > the sort type stuff) and given the fact I'm a Postgres enthusiast I > asked my self: "my server is

Re: [HACKERS] CUDA Sorting

2012-02-15 Thread Gaetano Mendola
On 15/02/2012 23:11, Peter Geoghegan wrote: On 15 February 2012 20:00, Gaetano Mendola wrote: On 13/02/2012 19:48, Greg Stark wrote: I don't think we should be looking at either CUDA or OpenCL directly. We should be looking for a generic library that can target either and is well maintained a

Re: [HACKERS] CUDA Sorting

2012-02-15 Thread Gaetano Mendola
On 15/02/2012 23:11, Peter Geoghegan wrote: On 15 February 2012 20:00, Gaetano Mendola wrote: On 13/02/2012 19:48, Greg Stark wrote: I don't think we should be looking at either CUDA or OpenCL directly. We should be looking for a generic library that can target either and is well maintained a

Re: [HACKERS] CUDA Sorting

2012-02-15 Thread Peter Geoghegan
On 15 February 2012 20:00, Gaetano Mendola wrote: > On 13/02/2012 19:48, Greg Stark wrote: >> >> I don't think we should be looking at either CUDA or OpenCL directly. >> We should be looking for a generic library that can target either and >> is well maintained and actively developed. Any GPU code

Re: [HACKERS] CUDA Sorting

2012-02-15 Thread Gaetano Mendola
On 13/02/2012 19:48, Greg Stark wrote: I don't think we should be looking at either CUDA or OpenCL directly. We should be looking for a generic library that can target either and is well maintained and actively developed. Any GPU code we write ourselves would rapidly be overtaken by changes in th

Re: [HACKERS] CUDA Sorting

2012-02-15 Thread Gaetano Mendola
On 13/02/2012 19:48, Greg Stark wrote: I don't think we should be looking at either CUDA or OpenCL directly. We should be looking for a generic library that can target either and is well maintained and actively developed. Any GPU code we write ourselves would rapidly be overtaken by changes in th

Re: [HACKERS] CUDA Sorting

2012-02-15 Thread Marti Raudsepp
On Mon, Feb 13, 2012 at 20:48, Greg Stark wrote: > I don't think we should be looking at either CUDA or OpenCL directly. > We should be looking for a generic library that can target either and > is well maintained and actively developed. I understand your point about using some external library f

Re: [HACKERS] CUDA Sorting

2012-02-14 Thread Gaetano Mendola
On 13/02/2012 08:26, Greg Smith wrote: On 02/11/2012 08:14 PM, Gaetano Mendola wrote: The trend is to have server capable of running CUDA providing GPU via external hardware (PCI Express interface with PCI Express switches), look for example at PowerEdge C410x PCIe Expansion Chassis from DELL.

Re: [HACKERS] CUDA Sorting

2012-02-13 Thread Gaetano Mendola
On Feb 13, 2012 7:49 p.m., "Greg Stark" wrote: > > I don't think we should be looking at either CUDA or OpenCL directly. > We should be looking for a generic library that can target either and > is well maintained and actively developed. Any GPU code we write > ourselves would rapidly be overtaken

Re: [HACKERS] CUDA Sorting

2012-02-13 Thread Greg Stark
I don't think we should be looking at either CUDA or OpenCL directly. We should be looking for a generic library that can target either and is well maintained and actively developed. Any GPU code we write ourselves would rapidly be overtaken by changes in the hardware and innovations in parallel al

Re: [HACKERS] CUDA Sorting

2012-02-13 Thread Gaetano Mendola
On Feb 13, 2012 11:39 a.m., "Kohei KaiGai" wrote: > > 2012/2/13 Greg Smith : > > On 02/11/2012 08:14 PM, Gaetano Mendola wrote: > >> > >> The trend is to have server capable of running CUDA providing GPU via > >> external hardware (PCI Express interface with PCI Express switches), look > >> for ex

Re: [HACKERS] CUDA Sorting

2012-02-13 Thread Kohei KaiGai
2012/2/13 Greg Smith : > On 02/11/2012 08:14 PM, Gaetano Mendola wrote: >> >> The trend is to have server capable of running CUDA providing GPU via >> external hardware (PCI Express interface with PCI Express switches), look >> for example at PowerEdge C410x PCIe Expansion Chassis from DELL. > > >

Re: [HACKERS] CUDA Sorting

2012-02-12 Thread Greg Smith
On 02/11/2012 08:14 PM, Gaetano Mendola wrote: The trend is to have server capable of running CUDA providing GPU via external hardware (PCI Express interface with PCI Express switches), look for example at PowerEdge C410x PCIe Expansion Chassis from DELL. The C410X adds 16 PCIe slots to a serv

Re: [HACKERS] CUDA Sorting

2012-02-12 Thread Gaetano Mendola
On 12/02/2012 13:13, Oleg Bartunov wrote: I'm wondering if CUDA will win in geomentry operations, for example, tesing point <@ complex_polygon I'm not sure if the algorithm you mentioned can be implemented in terms of vector algebra, blas, etc. It's plenty of geometry operations implemented i

Re: [HACKERS] CUDA Sorting

2012-02-12 Thread Oleg Bartunov
I'm wondering if CUDA will win in geomentry operations, for example, tesing point <@ complex_polygon Oleg On Sun, 12 Feb 2012, Gaetano Mendola wrote: On 19/09/2011 16:36, Greg Smith wrote: On 09/19/2011 10:12 AM, Greg Stark wrote: With the GPU I'm curious to see how well it handles multiple p

Re: [HACKERS] CUDA Sorting

2012-02-11 Thread Gaetano Mendola
On 19/09/2011 16:36, Greg Smith wrote: On 09/19/2011 10:12 AM, Greg Stark wrote: With the GPU I'm curious to see how well it handles multiple processes contending for resources, it might be a flashy feature that gets lots of attention but might not really be very useful in practice. But it would

Re: [HACKERS] CUDA Sorting

2012-02-11 Thread Gaetano Mendola
On 19/09/2011 21:41, PostgreSQL - Hans-Jürgen Schönig wrote: On Sep 19, 2011, at 5:16 PM, Tom Lane wrote: Greg Stark writes: That said, to help in the case I described you would have to implement the tapesort algorithm on the GPU as well. I think the real problem would be that we are seldo

Re: [HACKERS] CUDA Sorting

2011-09-27 Thread Vitor Reus
Hey hackers, I'm still having problems reading the values of the columns in tuplesort.c, in order to understand how to port this to CUDA. Should I use the heap_getattr macro to read them? 2011/9/24 Hannu Krosing > On Mon, 2011-09-19 at 10:36 -0400, Greg Smith wrote: > > On 09/19/2011 10:12 AM,

Re: [HACKERS] CUDA Sorting

2011-09-24 Thread Hannu Krosing
On Mon, 2011-09-19 at 10:36 -0400, Greg Smith wrote: > On 09/19/2011 10:12 AM, Greg Stark wrote: > > With the GPU I'm curious to see how well > > it handles multiple processes contending for resources, it might be a > > flashy feature that gets lots of attention but might not really be > > very use

Re: [HACKERS] CUDA Sorting

2011-09-24 Thread Hannu Krosing
On Mon, 2011-09-19 at 15:12 +0100, Greg Stark wrote: > On Mon, Sep 19, 2011 at 1:11 PM, Vitor Reus wrote: > > Since I'm new to pgsql development, I replaced the code of pgsql > > qsort_arg to get used with the way postgres does the sort. The problem > > is that I can't use the qsort_arg_comparator

Re: [HACKERS] CUDA Sorting

2011-09-20 Thread Nulik Nol
> > I already did some benchmarks with GPU sorting (not in pgsql), and > measured total sort times, copy bandwidth and energy usage, and got > some exciting results: Was that qsort implementation on CPU cache friendly and optimized for SSE ? To make a fair comparison you have to take the best CPU i

Re: [HACKERS] CUDA Sorting

2011-09-20 Thread Florian Pflug
On Sep19, 2011, at 19:46 , Stephen Frost wrote: > I agree that it'd be interesting to do, but I share Lord Stark's > feelings about the challenges and lack of potential gain- it's a very > small set of queries that would benefit from this. You need to be > working with enough data to make the cost

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Cédric Villemain
2011/9/19 Greg Smith : > On 09/19/2011 10:53 AM, Thom Brown wrote: >> >> But couldn't that also be seen as a chicken/egg situation? > > > The chicken/egg problem here is a bit deeper than just "no one offers GPUs > because no one wants them" on server systems.  One of the reasons there > aren't mor

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread PostgreSQL - Hans-Jürgen Schönig
On Sep 19, 2011, at 5:16 PM, Tom Lane wrote: > Greg Stark writes: >> That said, to help in the case I described you would have to implement >> the tapesort algorithm on the GPU as well. > > I think the real problem would be that we are seldom sorting just the > key values. If you have to push

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Greg Smith
On 09/19/2011 10:53 AM, Thom Brown wrote: But couldn't that also be seen as a chicken/egg situation? The chicken/egg problem here is a bit deeper than just "no one offers GPUs because no one wants them" on server systems. One of the reasons there aren't more GPUs in typical database server

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Stephen Frost
* Thom Brown (t...@linux.com) wrote: > But nVidia does produce a non-graphics-oriented GPGPU line called > Tesla dedicated to such processing. Just as a side-note, I've got a couple Tesla's that aren't doing terribly much at the moment and they're in a Linux 'server'-type box from Penguin computin

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Christopher Browne
On Mon, Sep 19, 2011 at 10:36 AM, Greg Smith wrote: > Intel's next generation Ivy Bridge chipset, expected for the spring of 2012, > is going to add support for OpenCL to the built-in motherboard GPU.  We may > eventually see that trickle into the server hardware side of things too. Note that Ama

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Nulik Nol
On Mon, Sep 19, 2011 at 7:11 AM, Vitor Reus wrote: > Hello everyone, > > I'm implementing a CUDA based sorting on PostgreSQL, and I believe it > can improve the ORDER BY statement performance in 4 to 10 times. I > already have a generic CUDA sort that performs around 10 times faster > than std qso

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Vitor Reus
2011/9/19 Thom Brown > Is your aim to have this committed into core PostgreSQL, or just for > your own version?  If it's the former, I don't anticipate any > enthusiasm from the hacker community. This is a research thesis and I'm not confident to commit it on the core just by myself. I will, howe

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Tom Lane
Greg Stark writes: > That said, to help in the case I described you would have to implement > the tapesort algorithm on the GPU as well. I think the real problem would be that we are seldom sorting just the key values. If you have to push the tuples through the GPU too, your savings are going to

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Thom Brown
On 19 September 2011 16:10, Thom Brown wrote: > On 19 September 2011 15:54, Greg Stark wrote: >> On Mon, Sep 19, 2011 at 3:36 PM, Greg Smith wrote: >>> The main problem here is that the sort of hardware commonly used for >>> production database servers doesn't have any serious enough GPU to supp

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Thom Brown
On 19 September 2011 15:54, Greg Stark wrote: > On Mon, Sep 19, 2011 at 3:36 PM, Greg Smith wrote: >> The main problem here is that the sort of hardware commonly used for >> production database servers doesn't have any serious enough GPU to support >> CUDA/OpenCL available > > Of course that coul

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Greg Stark
On Mon, Sep 19, 2011 at 3:36 PM, Greg Smith wrote: > The main problem here is that the sort of hardware commonly used for > production database servers doesn't have any serious enough GPU to support > CUDA/OpenCL available Of course that could change if adding a GPU would help Postgres... I would

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Thom Brown
On 19 September 2011 15:36, Greg Smith wrote: > On 09/19/2011 10:12 AM, Greg Stark wrote: >> >> With the GPU I'm curious to see how well >> it handles multiple processes contending for resources, it might be a >> flashy feature that gets lots of attention but might not really be >> very useful in

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Greg Smith
On 09/19/2011 10:12 AM, Greg Stark wrote: With the GPU I'm curious to see how well it handles multiple processes contending for resources, it might be a flashy feature that gets lots of attention but might not really be very useful in practice. But it would be very interesting to see. The m

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Greg Stark
On Mon, Sep 19, 2011 at 1:11 PM, Vitor Reus wrote: > Since I'm new to pgsql development, I replaced the code of pgsql > qsort_arg to get used with the way postgres does the sort. The problem > is that I can't use the qsort_arg_comparator comparator function on > GPU, I need to implement my own. I

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Thom Brown
On 19 September 2011 14:32, Vitor Reus wrote: > 2011/9/19 Thom Brown : >> On 19 September 2011 13:11, Vitor Reus wrote: >>> Hello everyone, >>> >>> I'm implementing a CUDA based sorting on PostgreSQL, and I believe it >>> can improve the ORDER BY statement performance in 4 to 10 times. I >>> alre

Re: [HACKERS] CUDA Sorting

2011-09-19 Thread Thom Brown
On 19 September 2011 13:11, Vitor Reus wrote: > Hello everyone, > > I'm implementing a CUDA based sorting on PostgreSQL, and I believe it > can improve the ORDER BY statement performance in 4 to 10 times. I > already have a generic CUDA sort that performs around 10 times faster > than std qsort. I

[HACKERS] CUDA Sorting

2011-09-19 Thread Vitor Reus
Hello everyone, I'm implementing a CUDA based sorting on PostgreSQL, and I believe it can improve the ORDER BY statement performance in 4 to 10 times. I already have a generic CUDA sort that performs around 10 times faster than std qsort. I also managed to load CUDA into pgsql. Since I'm new to p