[PERFORM] Avoiding Refreezing XIDs Repeatedly

2015-02-09 Thread bkrug
For a large database with lots of activity (transactions), the XIDs are very often re-frozen by AutoVacuum. Even when autovacuum_freeze_max_age is set to 2 billion, the XIDs can wrap every couple of days on an active database. This causes unnecessary changes to otherwise unmodified files and

Re: [PERFORM] Avoiding Refreezing XIDs Repeatedly

2015-02-09 Thread Matheus de Oliveira
On Mon, Feb 9, 2015 at 1:58 PM, bkrug bk...@usatech.com wrote: Couldn't postgres reserve a special XID that is never available for normal transactions but that indicates that any transaction can see it because it is so old? Then instead of constantly having to freeze old XIDs each time the

Re: [PERFORM] Avoiding Refreezing XIDs Repeatedly

2015-02-09 Thread Alvaro Herrera
Matheus de Oliveira wrote: On Mon, Feb 9, 2015 at 1:58 PM, bkrug bk...@usatech.com wrote: Couldn't postgres reserve a special XID that is never available for normal transactions but that indicates that any transaction can see it because it is so old? Then instead of constantly having to

Re: [PERFORM] Avoiding Refreezing XIDs Repeatedly

2015-02-09 Thread bkrug
Matheus de Oliveira wrote It changed in recent versions (9.3 or 9.4, I don't recall exactly which) and moved to tuple header, but what you described is exactly what was done, the xid was 2. Should the relfrozenxid of pg_class then equal 2 for very old and already vacuumed tables? Because that

Re: [PERFORM] Avoiding Refreezing XIDs Repeatedly

2015-02-09 Thread Alvaro Herrera
bkrug wrote: The problem I'm facing is that I have many large (several GB) tables that are not being changed (they are several days old) but auto-vacuum keeps scanning and updating them every time the xid wraps around and thus my rsync back-up process sees that the disk files have changed and

Re: [PERFORM] Avoiding Refreezing XIDs Repeatedly

2015-02-09 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: bkrug wrote: The problem I'm facing is that I have many large (several GB) tables that are not being changed (they are several days old) but auto-vacuum keeps scanning and updating them every time the xid wraps around and thus my rsync back-up

[PERFORM] Survey: Max TPS you've ever seen

2015-02-09 Thread Luis Antonio Dias de Sá Junior
Hi, A survay: with pgbench using TPS-B, what is the maximum TPS you're ever seen? For me: 12000 TPS. -- Luis Antonio Dias de Sá Junior

[PERFORM] Re: Migrating a FoxPro system and would like input on the best way to achieve optimal performance

2015-02-09 Thread David G Johnston
TonyS wrote Then each client has files within their own directory to keep the size of the tables manageable. Each client has 165 tables. These tables are all the same definition across the different groups. I have considered partitioning tables, but if I am correct that would result in

Re: [PERFORM] Avoiding Refreezing XIDs Repeatedly

2015-02-09 Thread Matheus de Oliveira
On Mon, Feb 9, 2015 at 4:45 PM, bkrug bk...@usatech.com wrote: Should the relfrozenxid of pg_class then equal 2 for very old and already vacuumed tables? Because that is not what I am seeing. hm... You meant in the entire table? Like an static table? Then no, it is done tuple by tuple only.

Re: [PERFORM] Avoiding Refreezing XIDs Repeatedly

2015-02-09 Thread Alvaro Herrera
bkrug wrote: Matheus de Oliveira wrote It changed in recent versions (9.3 or 9.4, I don't recall exactly which) and moved to tuple header, but what you described is exactly what was done, the xid was 2. Should the relfrozenxid of pg_class then equal 2 for very old and already vacuumed

Re: [ADMIN] [PERFORM] empty string Vs NULL

2015-02-09 Thread sridhar bamandlapally
In application code is while inserting/updating: INSERT/UPDATE into ... ( '' ) - which is empty string in PG, and in Oracle its NULL while selecting: SELECT ... WHERE column IS NULL / NOT NULL the issue is, while DML its empty string and while SELECT its comparing with NULL On Mon, Feb 9,

[PERFORM] Poor performance when deleting from entity-attribute-value type master-table

2015-02-09 Thread Andreas Joseph Krogh
Hi all.   Using PG-9.4.0 I'm seeing this trying to delete from an entity-master table:   *# explain analyze delete from onp_crm_entity where entity_id IN (select tmp.delivery_id from temp_delete_delivery_id tmp);     QUERY

[PERFORM] empty string Vs NULL

2015-02-09 Thread sridhar bamandlapally
Hi All We are testing our Oracle compatible business applications on PostgreSQL database, the issue we are facing is empty string Vs NULL In Oracle '' (empty string) and NULL are treated as NULL but, in PostgreSQL '' empty string not treated as NULL I need some *implicit* way in PostgreSQL

Re: [PERFORM] empty string Vs NULL

2015-02-09 Thread Pavel Stehule
Hi 2015-02-09 12:22 GMT+01:00 sridhar bamandlapally sridhar@gmail.com: Hi All We are testing our Oracle compatible business applications on PostgreSQL database, the issue we are facing is empty string Vs NULL In Oracle '' (empty string) and NULL are treated as NULL but, in

[PERFORM] Migrating a FoxPro system and would like input on the best way to achieve optimal performance

2015-02-09 Thread TonyS
I have a system that I am needing to convert from FoxPro files being accessed with DAO to PostgreSQL. This system serves 1,000 clients and will be expanding to 2,000 within the next 18 months. The current system has a directory with files that contain information of a global nature such as the

Re: [PERFORM] Poor performance when deleting from entity-attribute-value type master-table

2015-02-09 Thread Jerry Sievers
Andreas Joseph Krogh andr...@visena.com writes: Hi all. Using PG-9.4.0 I'm seeing this trying to delete from an entity-master table: *# explain analyze delete from onp_crm_entity where entity_id IN (select tmp.delivery_id from temp_delete_delivery_id tmp); QUERY PLAN

Re: [PERFORM] Survey: Max TPS you've ever seen

2015-02-09 Thread Gavin Flower
On 10/02/15 08:30, Luis Antonio Dias de Sá Junior wrote: Hi, A survay: with pgbench using TPS-B, what is the maximum TPS you're ever seen? For me: 12000 TPS. -- Luis Antonio Dias de Sá Junior Important to specify: 1. O/S 2. version of PostgreSQL 3. PostgreSQL configuration 4. hardware

Re: [PERFORM] Poor performance when deleting from entity-attribute-value type master-table

2015-02-09 Thread Andreas Joseph Krogh
På mandag 09. februar 2015 kl. 22:36:55, skrev Igor Neyman iney...@perceptron.com mailto:iney...@perceptron.com:     From: pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance-ow...@postgresql.org]On Behalf Of Andreas Joseph Krogh Sent: Monday, February 09, 2015 4:13 PM To:

Re: [PERFORM] Poor performance when deleting from entity-attribute-value type master-table

2015-02-09 Thread Josh Berkus
On 02/09/2015 01:12 PM, Andreas Joseph Krogh wrote: Are there any ways around this or do people simply avoid having FKs in schemas like this? Don't use EAV. It's a bad design pattern, especially for you, and you've just discovered one of the reasons why. (In fact, I am just today dismantling

Re: [PERFORM] Poor performance when deleting from entity-attribute-value type master-table

2015-02-09 Thread Igor Neyman
From: pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance-ow...@postgresql.org] On Behalf Of Andreas Joseph Krogh Sent: Monday, February 09, 2015 4:13 PM To: pgsql-performance@postgresql.org Subject: [PERFORM] Poor performance when deleting from entity-attribute-value type

Re: [PERFORM] empty string Vs NULL

2015-02-09 Thread Nikolas Everett
Its been a while since I really worked with Postgres, but could you write a trigger to convert empty string to null on save? You'd have to carefully apply it everywhere but it'd get you the searching for null finds empty. If that is all you do the you've got it. Essentially, there isn't a switch

Re: [ADMIN] [PERFORM] empty string Vs NULL

2015-02-09 Thread Marc Mamin
Hi 2015-02-09 12:22 GMT+01:00 sridhar bamandlapally sridhar@gmail.com: Hi All We are testing our Oracle compatible business applications on PostgreSQL database, the issue we are facing is empty string Vs NULL In Oracle '' (empty string) and NULL are treated as NULL

Re: [PERFORM] Avoiding Refreezing XIDs Repeatedly

2015-02-09 Thread bkrug
The problem I'm facing is that I have many large (several GB) tables that are not being changed (they are several days old) but auto-vacuum keeps scanning and updating them every time the xid wraps around and thus my rsync back-up process sees that the disk files have changed and must copy them.

Re: [PERFORM] [ADMIN] empty string Vs NULL

2015-02-09 Thread Geoff Winkless
On 9 February 2015 at 11:22, sridhar bamandlapally sridhar@gmail.com wrote: the issue we are facing is empty string Vs NULL In Oracle '' (empty string) and NULL are treated as NULL but, in PostgreSQL '' empty string not treated as NULL I need some *implicit* way in PostgreSQL where

Re: [PERFORM] [ADMIN] empty string Vs NULL

2015-02-09 Thread Geoff Winkless
On 9 February 2015 at 12:48, Geoff Winkless pgsqlad...@geoff.dj wrote: Oracle's own documentation suggests that developers should not rely on this behaviour since it may change in the future. ​ Just in case you're looking for it: