Re: [GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread rob stone
I think I need to ask more specific way. I have a table say `table1`, where I feed data from different CSV files. Now suppose I have inserted N records to my table `table1` from csv file `c1`. This is ok, next time when again I am importing from a different CSV file say `c2` to `table1`, I

Re: [GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread Arup Rakshit
On Sunday, May 24, 2015 07:24:41 AM you wrote: On 05/24/2015 04:55 AM, Arup Rakshit wrote: On Sunday, May 24, 2015 02:52:47 PM you wrote: On Sun, 2015-05-24 at 16:56 +0630, Arup Rakshit wrote: Hi, I am copying the data from a CSV file to a Table using COPY command. But one thing that

Re: [GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread Oliver Elphick
On Sun, 2015-05-24 at 16:56 +0630, Arup Rakshit wrote: Hi, I am copying the data from a CSV file to a Table using COPY command. But one thing that I got stuck, is how to skip duplicate records while copying from CSV to tables. By looking at the documentation, it seems, Postgresql don't have

Re: [GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread Adrian Klaver
On 05/24/2015 04:55 AM, Arup Rakshit wrote: On Sunday, May 24, 2015 02:52:47 PM you wrote: On Sun, 2015-05-24 at 16:56 +0630, Arup Rakshit wrote: Hi, I am copying the data from a CSV file to a Table using COPY command. But one thing that I got stuck, is how to skip duplicate records while

[GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread Arup Rakshit
Hi, I am copying the data from a CSV file to a Table using COPY command. But one thing that I got stuck, is how to skip duplicate records while copying from CSV to tables. By looking at the documentation, it seems, Postgresql don't have any inbuilt too to handle this with copy command. By

Re: [GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread Oliver Elphick
On Sun, 2015-05-24 at 18:25 +0630, Arup Rakshit wrote: Assuming you are using Unix, or can install Unix tools, run the input files through sort -u before passing them to COPY. Oliver Elphick I think I need to ask more specific way. I have a table say `table1`, where

Re: [GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread Adrian Klaver
On 05/24/2015 06:24 AM, Arup Rakshit wrote: On Sunday, May 24, 2015 07:24:41 AM you wrote: On 05/24/2015 04:55 AM, Arup Rakshit wrote: On Sunday, May 24, 2015 02:52:47 PM you wrote: On Sun, 2015-05-24 at 16:56 +0630, Arup Rakshit wrote: Hi, I am copying the data from a CSV file to a Table

Re: [GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread Arup Rakshit
On Sunday, May 24, 2015 02:52:47 PM you wrote: On Sun, 2015-05-24 at 16:56 +0630, Arup Rakshit wrote: Hi, I am copying the data from a CSV file to a Table using COPY command. But one thing that I got stuck, is how to skip duplicate records while copying from CSV to tables. By looking

Re: [GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread Francisco Olarte
Hi Arup On Sun, May 24, 2015 at 12:26 PM, Arup Rakshit arupraks...@rocketmail.com wrote: I am copying the data from a CSV file to a Table using COPY command. But one thing that I got stuck, is how to skip duplicate records while copying from CSV to tables. By looking at the documentation, it

Re: [GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread Arup Rakshit
On Sunday, May 24, 2015 07:52:43 AM you wrote: Is it if the entire row is duplicated? It is entire row. So, Olivers second solution. I have done this : columns_t1 = self.singleton_class.fields.map { |f| t1.#{f} }.join(,) columns_t2 = self.singleton_class.fields.map { |f| t2.#{f}

Re: [GENERAL] How to skip duplicate records while copying from CSV to table in Postgresql using COPY

2015-05-24 Thread Scott Marlowe
On Sun, May 24, 2015 at 4:26 AM, Arup Rakshit arupraks...@rocketmail.com wrote: Hi, I am copying the data from a CSV file to a Table using COPY command. But one thing that I got stuck, is how to skip duplicate records while copying from CSV to tables. By looking at the documentation, it