Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-12 Thread Marc Mamin
Thank Tom, I understand that the rationale behind choosing to create a new table from distinct records is that, since both approaches need full table scans, selecting distinct records is faster (and seems more straight forward) than finding/deleting duplicates; Hi, on a large table you may get

[GENERAL] anyone using oid2name?

2014-12-12 Thread Peter Eisentraut
pgsql-hackers are discussing some housekeeping in contrib. Is anyone using the oid2name tool? Otherwise, we might deprecate and eventually remove it. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription:

Re: [GENERAL] anyone using oid2name?

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 09:18:53AM -0500, Peter Eisentraut wrote: pgsql-hackers are discussing some housekeeping in contrib. Is anyone using the oid2name tool? Otherwise, we might deprecate and eventually remove it. Uh, if we remove it, what tool does someone use from the command-line to

Re: [GENERAL] anyone using oid2name?

2014-12-12 Thread Atri Sharma
On Friday, December 12, 2014, Bruce Momjian br...@momjian.us wrote: On Fri, Dec 12, 2014 at 09:18:53AM -0500, Peter Eisentraut wrote: pgsql-hackers are discussing some housekeeping in contrib. Is anyone using the oid2name tool? Otherwise, we might deprecate and eventually remove it.

Re: Fwd: Fwd: [GENERAL] Problem with pg_dump and decimal mark

2014-12-12 Thread Eric Svenson
Hi Adrian, so finally I have a workaround which is ok for me. When I seperate the tables and the data (using the -a and -s switch from pg_dump) into 2 sql backup files, everything works ok on the problem-VM. I try to investigate further in the coming weeks, I´m on holiday next week. Regards and

Re: [GENERAL] anyone using oid2name?

2014-12-12 Thread Peter Eisentraut
On 12/12/14 9:25 AM, Bruce Momjian wrote: On Fri, Dec 12, 2014 at 09:18:53AM -0500, Peter Eisentraut wrote: pgsql-hackers are discussing some housekeeping in contrib. Is anyone using the oid2name tool? Otherwise, we might deprecate and eventually remove it. Uh, if we remove it, what tool

Re: [GENERAL] anyone using oid2name?

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 09:42:48AM -0500, Peter Eisentraut wrote: On 12/12/14 9:25 AM, Bruce Momjian wrote: On Fri, Dec 12, 2014 at 09:18:53AM -0500, Peter Eisentraut wrote: pgsql-hackers are discussing some housekeeping in contrib. Is anyone using the oid2name tool? Otherwise, we

Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-12 Thread Daniel Begin
Seems promising but could you provide me a reference to PostgreSQL documentation regarding this a%8=* feature? Best Daniel -Original Message- From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of Marc Mamin Sent: December-12-14 06:41 To: Daniel

Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-12 Thread John McKown
On Fri, Dec 12, 2014 at 9:57 AM, Daniel Begin jfd...@hotmail.com wrote: Seems promising but could you provide me a reference to PostgreSQL documentation regarding this a%8=* feature? Best ​% is the modulus operator. Assuming a is an integer (I don't remember), then doing 8 selects of a

Re: [GENERAL] anyone using oid2name?

2014-12-12 Thread Jeff Janes
On Fri, Dec 12, 2014 at 6:18 AM, Peter Eisentraut pete...@gmx.net wrote: pgsql-hackers are discussing some housekeeping in contrib. Is anyone using the oid2name tool? Otherwise, we might deprecate and eventually remove it. I use it occasionally (but really dislike it) during testing and

Re: [GENERAL] anyone using oid2name?

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 09:18:01AM -0800, Jeff Janes wrote: On Fri, Dec 12, 2014 at 6:18 AM, Peter Eisentraut pete...@gmx.net wrote: pgsql-hackers are discussing some housekeeping in contrib. Is anyone using the oid2name tool? Otherwise, we might deprecate and eventually

[GENERAL] Re: Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-12 Thread David G Johnston
John McKown wrote I don't know, myself, why this would be faster. But I'm not any kind of a PostgreSQL expert either. It is faster because PostgreSQL does not have native parallelism. By using a%n in a where clause you can start n separate sessions and choose a different value of n for each

Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-12 Thread Marc Mamin
Seems promising but could you provide me a reference to PostgreSQL documentation regarding this a%8=* feature? Best % is the modulus operator. Assuming a is an integer (I don't remember), then doing 8 selects of a modulus 8 = for each of the possible results (0..7)? will each

Re: [GENERAL] pgxs question - linking c-functions to external libraries

2014-12-12 Thread Alan Nilsson
I recently had need to do the same thing and I am having no luck. Admittedly, I am not too keen on the postgres build setup and have not debugged this extensively, but rather hoped there was an easy answer up front. That said…. I am trying to link libuuid into a custom extension, here is my

Re: [GENERAL] pgxs question - linking c-functions to external libraries

2014-12-12 Thread Tom Lane
Alan Nilsson anils...@apple.com writes: I am trying to link libuuid into a custom extension, here is my make file (building PG 9.3.5 on CentOS 6.5 (GCC 4.4.7) fwiw): MODULES = aitpowerpg EXTENSION = aitpowerpg DATA = aitpowerpg--1.0.sql SHLIB_LINK += -luuid ifdef USE_PGXS PG_CONFIG =

Re: [GENERAL] pgxs question - linking c-functions to external libraries

2014-12-12 Thread Alan Nilsson
Awesome - thanks Tom, works. alan On Dec 12, 2014, at 1:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alan Nilsson anils...@apple.com writes: I am trying to link libuuid into a custom extension, here is my make file (building PG 9.3.5 on CentOS 6.5 (GCC 4.4.7) fwiw): MODULES = aitpowerpg

Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-12 Thread Daniel Begin
Thank Marc (and all others) I knew that nothing was for free and understanding where the costs come from would provide me with some rationale to make my choice! However, I understand from your answer that there is no definitive approach to do it right at this time (considering my specific

[GENERAL] function indexes, index only scan and sorting

2014-12-12 Thread Jonathan Vanasco
Can someone confirm a suspicion for me ? I have a moderately sized table (20+ columns, 3MM rows) that tracks tags. I have a lower(column) function index that is used simplify case-insensitive lookups. CREATE INDEX idx_tag_name_lower ON tag(lower(name)); I have a few complex queries

Re: [GENERAL] Removing duplicate records from a bulk upload (rationale behind selecting a method)

2014-12-12 Thread Jonathan Vanasco
On Dec 8, 2014, at 9:35 PM, Scott Marlowe wrote: select a,b,c into newtable from oldtable group by a,b,c; On pass, done. This is a bit naive, but couldn't this approach potentially be faster (depending on the system)? SELECT a, b, c INTO duplicate_records FROM ( SELECT a, b, c,

Re: [GENERAL] function indexes, index only scan and sorting

2014-12-12 Thread Tom Lane
Jonathan Vanasco postg...@2xlp.com writes: Am I correct in observing that the value of a function index can't be used for sorting ? No ... regression=# create table tt (f1 int, f2 text); CREATE TABLE regression=# create index on tt (lower(f2)); CREATE INDEX regression=# explain select * from

Re: [GENERAL] function indexes, index only scan and sorting

2014-12-12 Thread Jonathan Vanasco
On Dec 12, 2014, at 4:58 PM, Tom Lane wrote: regression=# create table tt (f1 int, f2 text); CREATE TABLE regression=# create index on tt (lower(f2)); CREATE INDEX regression=# explain select * from tt order by lower(f2); QUERY PLAN

Re: [GENERAL] function indexes, index only scan and sorting

2014-12-12 Thread Tom Lane
Jonathan Vanasco postg...@2xlp.com writes: Thank you so much for posting this test. I got a seq scan on my local machine, so I checked the version... still running 9.2.4. I tried it on production (which is 9.3.x) and got the same result as you. Hmm, well, I get the same result from 9.2.9,

[GENERAL] Re: [GENERAL] I did some testing of GIST/GIN vs BTree indexing…

2014-12-12 Thread Guyren Howe
On Dec 10, 2014, at 19:38 , Bruce Momjian br...@momjian.us wrote: Are you saying when you use a GIN index on a,b,c fields, you can do lookups on them independently, like 'c'? I was not aware that works, but it might. I know it doesn't work for traditional btree as the index is