Re: [GENERAL] Need for re-index after pg_upgrade

2015-06-29 Thread Andreas Joseph Krogh
På mandag 29. juni 2015 kl. 15:42:22, skrev Tom Lane t...@sss.pgh.pa.us mailto:t...@sss.pgh.pa.us: Andreas Joseph Krogh andr...@visena.com writes: In the man-page for pg_upgrade we see this: pg_upgrade will require a reindex if:   * an index is of type hash or GIN I think that probably

[GENERAL] Which replication is the best for our case ?

2015-06-29 Thread ben.play
Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about 10 000 lines with complex algorithms in PHP. Each time the cron runs, the website is almost down because some queries have to make an update on the FULL table...

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Adrian Klaver
On 06/29/2015 06:02 AM, ben.play wrote: Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about 10 000 lines with complex algorithms in PHP. Each time the cron runs, the website is almost down because some queries have to

Re: [GENERAL] Need for re-index after pg_upgrade

2015-06-29 Thread Tom Lane
Andreas Joseph Krogh andr...@visena.com writes: In the man-page for pg_upgrade we see this: pg_upgrade will require a reindex if: * an index is of type hash or GIN I think that probably refers to some version-specific upgrade situations; I can't see a reason why it would be true in general.

Re: [GENERAL] Need for re-index after pg_upgrade

2015-06-29 Thread Adrian Klaver
On 06/29/2015 06:42 AM, Tom Lane wrote: Andreas Joseph Krogh andr...@visena.com writes: In the man-page for pg_upgrade we see this: pg_upgrade will require a reindex if: * an index is of type hash or GIN I think that probably refers to some version-specific upgrade situations; I can't see a

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Adrian Klaver
On 06/29/2015 06:02 AM, ben.play wrote: Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about 10 000 lines with complex algorithms in PHP. Each time the cron runs, the website is almost down because some queries have to

[GENERAL] Need for re-index after pg_upgrade

2015-06-29 Thread Andreas Joseph Krogh
Hi all.   In the man-page for pg_upgrade we see this: pg_upgrade will require a reindex if: * an index is of type hash or GIN   A bit further up we see: All failure, rebuild, and reindex cases will be reported by pg_upgrade if they affect your installation; post-upgrade scripts to rebuild

Re: [GENERAL] pg_xlog on a hot_stanby slave

2015-06-29 Thread Stéphane Schildknecht
On 16/06/2015 10:55, Xavier 12 wrote: Hi everyone, Questions about pg_xlogs again... I have two Postgresql 9.1 servers in a master/slave stream replication (hot_standby). Psql01 (master) is backuped with Barman and pg_xlogs is correctly purged (archive_command is used). Hower, Psql02

Re: [GENERAL] pg_xlog on a hot_stanby slave

2015-06-29 Thread Xavier 12
On 29/06/2015 11:38, Stéphane Schildknecht wrote: On 16/06/2015 10:55, Xavier 12 wrote: Hi everyone, Questions about pg_xlogs again... I have two Postgresql 9.1 servers in a master/slave stream replication (hot_standby). Psql01 (master) is backuped with Barman and pg_xlogs is correctly

Re: [GENERAL] Correct place for feature requests

2015-06-29 Thread Merlin Moncure
On Thu, Jun 25, 2015 at 1:59 PM, Алексей Бережняк c...@irc.lg.ua wrote: I think that PostgreSQL is great RDBMS, but one important (for me) feature that it missing is case-insensitive identifier quotes ([table].[column]) like in Microsoft SQL Server. I know that there are double quotes, but

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Holger.Friedrich-Fa-Trivadis
Arthur Silva wrote on Monday, June 29, 2015 5:23 PM: Therefore, I'm asking if it's possible to duplicate my main database on a slave server in order to run these cron on this second server... then, replicate these changes on the main database (master).

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Arthur Silva
On Mon, Jun 29, 2015 at 10:02 AM, ben.play benjamin.co...@playrion.com wrote: Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about 10 000 lines with complex algorithms in PHP. Each time the cron runs, the website is

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Adrian Klaver
On 06/29/2015 08:23 AM, Arthur Silva wrote: On Mon, Jun 29, 2015 at 10:02 AM, ben.play benjamin.co...@playrion.com mailto:benjamin.co...@playrion.com wrote: Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about

Re: [GENERAL] Re: Get the difference between two timestamp cells but in a special format in PostgreSQL

2015-06-29 Thread Colin Lieberman
Check the formatting functions documentation: http://www.postgresql.org/docs/9.4/static/functions-formatting.html # select to_char( now() - '2015-06-27 14:33:24' , 'Yyears MMmonths DDdays HH:MI:SS.MS' ); to_char - 0years 00months 01days

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Jeff Janes
On Mon, Jun 29, 2015 at 6:02 AM, ben.play benjamin.co...@playrion.com wrote: Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about 10 000 lines with complex algorithms in PHP. Each time the cron runs, the website is

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread John R Pierce
On 6/29/2015 10:41 AM, Melvin Davidson wrote: I think it would help immensely if you provided details such as table_structure, indexes the actual UPDATE query and the reason all rows of the table must be updated. indeed, the whole model of massaging the entire database every 10 minutes is

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Melvin Davidson
I think it would help immensely if you provided details such as table_structure, indexes the actual UPDATE query and the reason all rows of the table must be updated. On Mon, Jun 29, 2015 at 1:15 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Jun 29, 2015 at 6:02 AM, ben.play

Re: [GENERAL] Inserting from multiple processes?

2015-06-29 Thread Francisco Olarte
Hi Dave: On Mon, Jun 29, 2015 at 6:32 AM, Dave Johansen davejohan...@gmail.com wrote: The issue is that the following uses 5 XIDs when I would only expect it to us 1: BEGIN; SELECT insert_test_no_dup('2015-01-01', 1, 1); END; I see. It appears that the unique violation that is caught

Re: [GENERAL] archive_timeout and WAL size

2015-06-29 Thread Adrian Klaver
On 06/29/2015 11:49 AM, Edson Richter wrote: Dear community, I'm using PostgreSQL 9.3.6 on Linux x64. Would sound a stupid questions, and sorry if it was already asked before: if I set the archive_timeout, and then I have them sent every minute (for example), are the files still 16MB in size,

Re: [GENERAL] archive_timeout and WAL size

2015-06-29 Thread Edson Richter
Thanks, Adrian. That's the reference I was looking for. Atenciosamente, Edson Carlos Ericksson Richter Em 29/06/2015 15:55, Adrian Klaver escreveu: On 06/29/2015 11:49 AM, Edson Richter wrote: Dear community, I'm using PostgreSQL 9.3.6 on Linux x64. Would sound a stupid questions, and

[GENERAL] plpgsql question: select into multiple variables ?

2015-06-29 Thread Day, David
Hi, Postgres version 9.3.9 What is wrong with my usage of the plpgsql select into concept I have a function to look into a calendar table to find the first and Last weekend date of a month. In this simplified concept function I end up with a NULL for first or last weekend variable.

[GENERAL] archive_timeout and WAL size

2015-06-29 Thread Edson Richter
Dear community, I'm using PostgreSQL 9.3.6 on Linux x64. Would sound a stupid questions, and sorry if it was already asked before: if I set the archive_timeout, and then I have them sent every minute (for example), are the files still 16MB in size, or are they truncated and sent in smaller

Re: [GENERAL] plpgsql question: select into multiple variables ?

2015-06-29 Thread Day, David
-Original Message- From: Adrian Klaver [mailto:adrian.kla...@aklaver.com] Sent: Monday, June 29, 2015 4:03 PM To: Day, David; pgsql-general@postgresql.org Subject: Re: [GENERAL] plpgsql question: select into multiple variables ? On 06/29/2015 12:07 PM, Day, David wrote: Hi,

Re: [GENERAL] plpgsql question: select into multiple variables ?

2015-06-29 Thread David G. Johnston
On Mon, Jun 29, 2015 at 4:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Adrian Klaver adrian.kla...@aklaver.com writes: On 06/29/2015 12:07 PM, Day, David wrote: What is wrong with my usage of the plpgsql select into concept I have a function to look into a calendar table to find the first

Re: [GENERAL] plpgsql question: select into multiple variables ?

2015-06-29 Thread Tom Lane
Adrian Klaver adrian.kla...@aklaver.com writes: On 06/29/2015 12:07 PM, Day, David wrote: What is wrong with my usage of the plpgsql select into concept I have a function to look into a calendar table to find the first and Last weekend date of a month. create or replace function

Re: [GENERAL] plpgsql question: select into multiple variables ?

2015-06-29 Thread Tom Lane
David G. Johnston david.g.johns...@gmail.com writes: On Mon, Jun 29, 2015 at 4:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: ... So what you wrote here is equivalent to SELECT MIN(CAL_DATE),MAX(CAL_DATE) ::date, last_weekend::date INTO first_weekend FROM sys.calendar ... ​Does it help to

Re: [GENERAL] serialization failure why?

2015-06-29 Thread Simon Riggs
On 29 June 2015 at 21:13, Kevin Grittner kgri...@ymail.com wrote: Simon Riggs si...@2ndquadrant.com wrote: On 17 June 2015 at 13:52, Kevin Grittner kgri...@ymail.com wrote: Filipe Pina filipe.p...@impactzero.pt wrote: if drop the foreign key constraint on stuff_ext table there are no

Re: [GENERAL] plpgsql question: select into multiple variables ?

2015-06-29 Thread Adrian Klaver
On 06/29/2015 12:07 PM, Day, David wrote: Hi, Postgres version 9.3.9 What is wrong with my usage of the plpgsql select into concept I have a function to look into a calendar table to find the first and Last weekend date of a month. In this simplified concept function I end up with a NULL

[GENERAL] WAL archive resend policy

2015-06-29 Thread Edson Richter
Dear all, Another question about WAR archiving: what is the resend policy if remote storage runs out of space? The failed archives will be resend automatically in future, or there is need for manual interation? Thanks, -- Atenciosamente, Edson Carlos Ericksson Richter -- Sent via

Re: [GENERAL] WAL archive resend policy

2015-06-29 Thread Joshua D. Drake
On 06/29/2015 12:49 PM, Edson Richter wrote: Dear all, Another question about WAR archiving: what is the resend policy if remote storage runs out of space? The failed archives will be resend automatically in future, or there is need for manual interation? They will be resent. Thanks,

Re: [GENERAL] Feature request: fsync and commit_delay options per database

2015-06-29 Thread Tom Lane
=?UTF-8?Q?Br=C3=A1ulio_Bhavamitra?= brauli...@gmail.com writes: Besides increasing it, it would be great to have these two options (fsync and commit_delay) per database, that is, valid only for databases configured with them. That would greatly speed up test running and still make the cluster

Re: [GENERAL] serialization failure why?

2015-06-29 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: On 17 June 2015 at 13:52, Kevin Grittner kgri...@ymail.com wrote: Filipe Pina filipe.p...@impactzero.pt wrote: if drop the foreign key constraint on stuff_ext table there are no failures at all… It is my recollection that we were excluding the

[GENERAL] Feature request: fsync and commit_delay options per database

2015-06-29 Thread Bráulio Bhavamitra
Hello all, After reading http://stackoverflow.com/questions/9407442/optimise-postgresql-for-fast-testing I've tried to use commit_delay to make commits really slow on a test environment. Unfortunetely, the maximum value is 100ms (100_000 microseconds). Besides increasing it, it would be great

Re: [GENERAL] Feature request: fsync and commit_delay options per database

2015-06-29 Thread Bráulio Bhavamitra
On Mon, Jun 29, 2015 at 7:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: =?UTF-8?Q?Br=C3=A1ulio_Bhavamitra?= brauli...@gmail.com writes: Besides increasing it, it would be great to have these two options (fsync and commit_delay) per database, that is, valid only for databases configured with them.

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Andy Erskine
no snapshot available .. i don't mind running basebackup once i've finished my test. So if someone could help with the steps to turn off replication and bring secondary up as a standalone db that would be great thanks. On 29 June 2015 at 15:58, John R Pierce pie...@hogranch.com wrote: On

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Andy Erskine
Thanks Jeff, I don't want a clone - i want to temporaily turn off replication (and therefore failover) and load a different db into the secondary which is now writable and run some tests. Then i will remove this db and run a basebackup to reinstate a copy of the master and turn on replication

Re: [GENERAL] create index on a field of udt

2015-06-29 Thread John R Pierce
On 6/28/2015 11:24 PM, Shujie Shang wrote: insert into test values (generate_series(1, 300), (1, 'hi')::info); explain select * from test where i.id http://i.id=1; the result is : seqscan does not every row of that match i.id = 1 ? try ... insert into test values (generate_series(1,

Re: [GENERAL] create index on a field of udt

2015-06-29 Thread Charles Clavadetscher
+1 create index on test (((i).id)); ANALYZE explain select * from test where (i).id = 8909; QUERY PLAN - Index Scan using test_id_idx on test (cost=0.43..8.45 rows=1 width=34) Index

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Andy Erskine
Cheers Micheal, So i don't want to touch the file as this will promote it too a master - i would just like it as a standalone db for this instance - i need to load a db and do some testing. After which i will delete the data directory and run the basebackup cmd and pull the db back from the

Re: [GENERAL] create index on a field of udt

2015-06-29 Thread Jeff Janes
On Sun, Jun 28, 2015 at 10:31 PM, Shujie Shang ssh...@pivotal.io wrote: Oh, I didn't explain my question well, actually I want to create an index on an udt in a table. e.g. create type info as (id int, name text); creat table test (i info); I want to run: create index myindex on test

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread John R Pierce
On 6/29/2015 12:06 AM, Jeff Janes wrote: But since you want a clone, what is the point of first setting up streaming, and then breaking it? Just use pg_basebackup to set up a clone directly, without ever having started streaming. It seems like you are just going to confuse yourself about

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread John R Pierce
On 6/28/2015 11:58 PM, Andy Erskine wrote: If i touch my trigger file and promote my secondary to a master - what effect will that have on the Master - will i need to make any changes on that side ? Will it still try and stream data across to the promoted secondary and just fill up the log

Re: [GENERAL] create index on a field of udt

2015-06-29 Thread Shujie Shang
Hi, I find a way to create index, I create a function returns the 'id' field of udt info, then I create index based on this function. e.g create type info as (id int, name text); creat table test (id int, i info); create or replace function getID(i info) returns int as $$ select $1.id $$

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Andy Erskine
If i touch my trigger file and promote my secondary to a master - what effect will that have on the Master - will i need to make any changes on that side ? Will it still try and stream data across to the promoted secondary and just fill up the log files with error messages ? On 29 June 2015 at

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Jeff Janes
On Sun, Jun 28, 2015 at 11:34 PM, Andy Erskine andy.ersk...@jds.net.au wrote: no snapshot available .. i don't mind running basebackup once i've finished my test. So if someone could help with the steps to turn off replication and bring secondary up as a standalone db that would be great

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Michael Paquier
On Mon, Jun 29, 2015 at 3:34 PM, Andy Erskine andy.ersk...@jds.net.au wrote: no snapshot available .. i don't mind running basebackup once i've finished my test. So if someone could help with the steps to turn off replication and bring secondary up as a standalone db that would be great.

Re: [GENERAL] pg_xlog on a hot_stanby slave

2015-06-29 Thread Xavier 12
On 19/06/2015 03:31, Sameer Kumar wrote: On Thu, 18 Jun 2015 15:17 Xavier 12 mania...@gmail.com mailto:mania...@gmail.com wrote: On 18/06/2015 04:00, Sameer Kumar wrote: On Wed, 17 Jun 2015 15:24 Xavier 12 mania...@gmail.com mailto:mania...@gmail.com wrote:

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Tomas Vondra
Hi, On 06/29/2015 09:27 AM, Andy Erskine wrote: Thanks Jeff, I don't want a clone - i want to temporaily turn off replication (and therefore failover) and load a different db into the secondary which is now writable and run some tests. Then i will remove this db and run a basebackup to

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Andy Erskine
agreed there is an element of risk. however a backup of the master will be carried out prior (ok there could potentially be a gap of data during downtime) unfortunately i have nothing else big enough (diskwise) to run my tests on. On 30 June 2015 at 15:47, Michael Paquier

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Andy Erskine
No i don't want a replica. I would like to reconfigure my streaming scenario into two standalone db's i don't want the Master to be effected in anyway and i want it running consistantly .. the secondary i want to reconfigure as a standalone to load a different db and do some testing. When i've

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Michael Paquier
On Tue, Jun 30, 2015 at 1:39 PM, Andy Erskine andy.ersk...@jds.net.au wrote: No i don't want a replica. I would like to reconfigure my streaming scenario into two standalone db's i don't want the Master to be effected in anyway and i want it running consistantly .. the secondary i want to

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Andy Erskine
Ok so a question i should have asked at the very beginning .. If i touch my trigger file - promoting the secondary to a master - will that in anyway effect the master thats already running ? IE no files on the master will change ? Then all i'll have to do is shutdown the secondary when i've

Re: [GENERAL] Turn off streaming replication - leaving Master running

2015-06-29 Thread Michael Paquier
On Tue, Jun 30, 2015 at 2:29 PM, Andy Erskine andy.ersk...@jds.net.au wrote: Ok so a question i should have asked at the very beginning .. If i touch my trigger file - promoting the secondary to a master - will that in anyway effect the master thats already running ? IE no files on the