[GENERAL] Success story full text search

2015-04-30 Thread Frank Langel
Hi, Does someone have a success story of using Postgres Full Search Capability with significant data, lets say 50-100 GB ? Any pointers would be much appreciated Thanks Frank

[GENERAL] some useless files may be left behind ?

2015-04-30 Thread Bo Thorbjørn Jensen
Hi All, We have a cluster on windows 2008 server PostgreSQL 9.1.15, compiled by Visual C++ build 1500, 64-bit After trying to create new databases concurrently npgsql endpoint reached default commandtimeout (20 seconds). This results in following log entries on the server: 2015-04-30 08:47:14

Re: [GENERAL] clearing of the transactions shown in pg_locks

2015-04-30 Thread Mitu Verma
Thanks Albe . It worked. But now there is one more issue which I am facing regarding this. I performed the following steps - 1. I stopped the script which is doing some operation( delete etc) in one of the tables. 2. SELECT pg_terminate_backend(proc pid). It has terminated the database

[GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Charlton Galvarino
Hello, psql family. psql (8.4.7) I am having mixed results for a warm standby. My warm standby appears to work for a couple of days and then refuses to process any more WAL files. I will see lovely entries like this in standby.log: removing /var/lib/pgsql/archives/000110E500E9 ...

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Charlton Galvarino
Fair enough. I should have thought a bit harder before airing my dirty laundry! -Original Message- From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general- ow...@postgresql.org] On Behalf Of Chris Mair Sent: Thursday, April 30, 2015 9:41 AM To: pgsql-general@postgresql.org

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Charlton Galvarino
So where does the rsync you mentioned previously fit into this? A cron runs on warm_standby to pull (and delete) the WAL's from the master. rsync -avz --progress --remove-sent-files master:/var/lib/pgsql/archives/ /var/lib/pgsql/archives/ I do not see an -l option in the 8.4.x version of

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Charlton Galvarino
Hmm, in newer versions of rsync --remove-sent-files has been replaced by -- remove-source-files, so I cannot test. Some searching found that --remove- sent-files will move/delete unfinished files. I would say that the above cron command is dangerous. To test, comment out the command and let the

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Charlton Galvarino
So what is your archive_command? archive_mode = on archive_command = 'cp -v %p /var/lib/pgsql/archives/%f' archive_timeout = 300 Where is the listing below from?: The below listing was warm_standby: /var/lib/pgsql/archives/ -rw--- 1 postgres postgres 16777216 Apr 29 05:07

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Adrian Klaver
On 04/30/2015 07:03 AM, Charlton Galvarino wrote: So what is your archive_command? archive_mode = on archive_command = 'cp -v %p /var/lib/pgsql/archives/%f' archive_timeout = 300 So where does the rsync you mentioned previously fit into this? Where is the listing below from?: The below

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Adrian Klaver
On 04/30/2015 07:42 AM, Charlton Galvarino wrote: Hmm, in newer versions of rsync --remove-sent-files has been replaced by -- remove-source-files, so I cannot test. Some searching found that --remove- sent-files will move/delete unfinished files. I would say that the above cron command is

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Adrian Klaver
On 04/30/2015 07:22 AM, Charlton Galvarino wrote: So where does the rsync you mentioned previously fit into this? A cron runs on warm_standby to pull (and delete) the WAL's from the master. rsync -avz --progress --remove-sent-files master:/var/lib/pgsql/archives/ /var/lib/pgsql/archives/

Re: [GENERAL] Partition Help

2015-04-30 Thread akshunj
Hi Jim, I'm not going the partitioning route because I want to, I'm just out of options at this point. As this table gets bigger, performance just gets worse over time. I wanted to try partitioning to see if it helps. Thanks for the tip, looking at the function again what you suggest makes

Re: [GENERAL] Partition Help

2015-04-30 Thread akshunj
Melvin, thanks. Syntax was indeed the problem there. Any idea how to pass the original query to the child table? My insert has 113 parameters passed in, but based on the constraint violations I am seeing, it seems they are not making. I suspect that: VALUE (NEW.*) does not pass in the original

[GENERAL] pgbench - prevent client from aborting on ERROR

2015-04-30 Thread Nicholson, Brad (Toronto, ON, CA)
Hi, Is there any way to do this? For context, I'm wanting to write a custom script in repeatable read isolation level. If I hit a serializable error, I don't want the client to abort, I want it to continue running transactions. Is that possible? thanks, Brad. -- Sent via pgsql-general

Re: [GENERAL] PostgreSQL HA config recommendations

2015-04-30 Thread William Dunn
Alex, Note that you should be weary of suggestions to make your replication synchronous. Synchronous replication is rarely used for this kind of use case (Cisco Jabber) where the most complete durability of the standby is not of the utmost concern (as it would be in a banking application). Not

Re: [GENERAL] newsfeed type query

2015-04-30 Thread Ladislav Lenart
On 30.4.2015 19:08, Jonathan Vanasco wrote: On Apr 29, 2015, at 6:50 PM, Jim Nasby wrote: Only because you're using UNION. Use UNION ALL instead. The difference between union and union all was negligible. the problem was in the subselect and the sheer size of the tables, even when we

Re: [GENERAL] PostgreSQL HA config recommendations

2015-04-30 Thread William Dunn
Alex, Note that you should be weary of suggestions to make your replication synchronous. Synchronous replication is rarely used for this kind of use case (Cisco Jabber) where the most complete durability of the standby is not of the utmost concern (as it would be in a banking application). Not

Re: [GENERAL] Partition Help

2015-04-30 Thread Melvin Davidson
Rick, I am glad I could help, but I am not quite sure you understand the purpose/use of the trigger to partition the table. The trigger merely decides which child should get the data, the query is not passed, only the data, To be more specific, if your appropriate child table were child_1000,

[GENERAL] how to read all physical rows (visible or not) from a heap

2015-04-30 Thread Qingqing Zhou
I recall once there is a GUC allowing us - basically force a SnapshotAny - what's current trick now? I need this only for debugging reading heap pages. Thanks, Qingqing -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription:

Re: [GENERAL] PostgreSQL HA config recommendations

2015-04-30 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Bucardo and pg_pool are both based on the idea of replaying SQL statements instead of replicating actual data. They have their uses, but I personally distrust that idea, especially for DR. Actually, Bucardo is very data-based, not

Re: [GENERAL] newsfeed type query

2015-04-30 Thread Jonathan Vanasco
On Apr 29, 2015, at 6:50 PM, Jim Nasby wrote: Only because you're using UNION. Use UNION ALL instead. The difference between union and union all was negligible. the problem was in the subselect and the sheer size of the tables, even when we could handle it as an index-only scan. On Apr

Re: [GENERAL] pgbench - prevent client from aborting on ERROR

2015-04-30 Thread Adrian Klaver
On 04/30/2015 11:36 AM, Nicholson, Brad (Toronto, ON, CA) wrote: Hi, Is there any way to do this? For context, I'm wanting to write a custom script in repeatable read isolation level. If I hit a serializable error, I don't want the client to abort, I want it to continue running

Re: [GENERAL] database split

2015-04-30 Thread Dave Owens
All that said, my guess is you're doing this to support horizontal scale-out, which means you'll probably need to do this more than once, and it'd presumably be nice for you and your customers if this didn't require downtime. I would look at having a way to create a partial replica using

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Charlton Galvarino
archiving run. Postgres will recycle WALs on its own when they are no longer needed. Or is there is some compelling reason you want to get rid of WALs? Ah. I didn't know that. I thought the cleanup was on me. Bonus! For more info take a look here:

[GENERAL] Removing and readding bdr nodes

2015-04-30 Thread Mathew Moon
Hi, I have a group of 5 bdr nodes and before we can move to production with them we must demonstrate that we can remove a node from the group and add that node back later. When I remove a node it stays in the bdr.bdr_nodes view with status 'k'. If I try to add that node back the node itself

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Charlton Galvarino
MASTER * archive_command = 'cp -v %p /var/lib/pgsql/archives/%f' * rsync files in [MASTER:/var/lib/pgsql/archives that are +10m and delete on them once sent] to [STANDBY:/var/lib/pgsql/archives] STANDBY * restore_command = 'pg_standby -d -s 2 -t /tmp/pgsql.trigger /var/lib/pgsql/archives

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Adrian Klaver
On 04/30/2015 05:22 PM, Charlton Galvarino wrote: MASTER * archive_command = 'cp -v %p /var/lib/pgsql/archives/%f' * rsync files in [MASTER:/var/lib/pgsql/archives that are +10m and delete on them once sent] to [STANDBY:/var/lib/pgsql/archives] STANDBY * restore_command = 'pg_standby

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Adrian Klaver
On 04/30/2015 01:49 PM, Charlton Galvarino wrote: archiving run. Postgres will recycle WALs on its own when they are no longer needed. Or is there is some compelling reason you want to get rid of WALs? Ah. I didn't know that. I thought the cleanup was on me. Bonus! For more info take a

[GENERAL] Finding new or modified rows since snapshot

2015-04-30 Thread Meel Velliste
My goal is to select rows that are new or have been modified since a given snapshot. I am doing it like this: SELECT * FROM my_table WHERE NOT txid_visible_in_snapshot(xmin::TEXT::BIGINT, '123456:123456:'::TXID_SNAPSHOT); On one hand, it seems to me that the txid_visible_in_snapshot function was

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Chris Mair
psql (8.4.7) Uhm the last update to 8.4 was 8.4.22: besides using an unsupported version, you're missing three and a half years of patches in 8.4.x :| Bye, Chris -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription:

Re: [GENERAL] psql 8 warm standby strong start, weak finish

2015-04-30 Thread Adrian Klaver
On 04/30/2015 06:31 AM, Charlton Galvarino wrote: Hello, psql family. psql (8.4.7) I am having mixed results for a warm standby. My warm standby appears to work for a couple of days and then refuses to process any more WAL files. I will see lovely entries like this in standby.log: removing