[PERFORM] Duplicate deletion optimizations

2012-01-06 Thread antoine
Hello, I've a table with approximately 50 million rows with a schema like this: id bigint NOT NULL DEFAULT nextval('stats_5mn'::regclass), t_value integer NOT NULL DEFAULT 0, t_record integer NOT NULL DEFAULT 0, output_id integer NOT NULL DEFAULT 0, count bigint NOT NULL

Re: [PERFORM] Duplicate deletion optimizations

2012-01-06 Thread antoine
On Fri, 06 Jan 2012 15:35:36 +0100, anto...@inaps.org wrote: Hello, I've a table with approximately 50 million rows with a schema like this: id bigint NOT NULL DEFAULT nextval('stats_5mn'::regclass), t_value integer NOT NULL DEFAULT 0, t_record integer NOT NULL DEFAULT 0,

Re: [PERFORM] Duplicate deletion optimizations

2012-01-06 Thread Samuel Gendler
On Fri, Jan 6, 2012 at 6:35 AM, anto...@inaps.org wrote: Hello, I've a table with approximately 50 million rows with a schema like this: id bigint NOT NULL DEFAULT nextval('stats_5mn'::regclass)**, t_value integer NOT NULL DEFAULT 0, t_record integer NOT NULL DEFAULT 0,

Re: [PERFORM] Duplicate deletion optimizations

2012-01-06 Thread Marc Eberhard
Hi Samuel! On 6 January 2012 20:02, Samuel Gendler sgend...@ideasculptor.com wrote: Have you considered doing the insert by doing a bulk insert into a temp table and then pulling rows that don't exist across to the final table in one query and updating rows that do exist in another query?  I

Re: [PERFORM] Duplicate deletion optimizations

2012-01-06 Thread Samuel Gendler
On Fri, Jan 6, 2012 at 12:22 PM, Marc Eberhard eberhar...@googlemail.comwrote: Hi Samuel! On 6 January 2012 20:02, Samuel Gendler sgend...@ideasculptor.com wrote: Have you considered doing the insert by doing a bulk insert into a temp table and then pulling rows that don't exist across to

Re: [PERFORM] Duplicate deletion optimizations

2012-01-06 Thread Marc Eberhard
On 6 January 2012 20:38, Samuel Gendler sgend...@ideasculptor.com wrote: On Fri, Jan 6, 2012 at 12:22 PM, Marc Eberhard eberhar...@googlemail.com wrote: On 6 January 2012 20:02, Samuel Gendler sgend...@ideasculptor.com wrote: Have you considered doing the insert by doing a bulk insert into a

Re: [PERFORM] Duplicate deletion optimizations

2012-01-06 Thread Misa Simic
hi, Maybe these thoughts could help 1) order by those three columns in your select min query could force index usage... 2) or DELETE FROM table WHERE EXISTS(SELECT id FROM table t WHERE t.id table.id AND t.col1 = table.col1 AND t.col2 = table.col2 AND col3 = table.col3) Sent from my