Re: [GENERAL] newsfeed type query

2015-04-29 Thread Ladislav Lenart
Hello. On 29.4.2015 01:57, Jonathan Vanasco wrote: Sorry, I was trying to ask something very abstract as I have similar situations on multiple groups of queries/tables (and they're all much more complex). I'm on pg 9.3 The relevant structure is: posting: id

Re: [GENERAL] Upgrading hot standbys

2015-04-29 Thread Magnus Hagander
On Wed, Apr 29, 2015 at 6:19 AM, Aaron Burnett aaron.burn...@us.dunnhumby.com wrote: Greetings, I'm in the process of upgrading PG 9.1.3 to 9.4.1 in the near future. I have several machines which each house unique databases. Each of those are replicated to a standby server with

Re: [GENERAL] clearing of the transactions shown in pg_locks

2015-04-29 Thread Mitu Verma
Hi, I have killed the script, but still the query is showing in pg_stat and pg_locks. Please help me how to clear the pg_locks from the transaction it is already holding, if there is something in PostgreSQL which can clear the pg_stat and pg_locks? root@s3bgwa31 # ps -efa| grep -i 28223

[GENERAL] Pg_bulkload and speed

2015-04-29 Thread Job
Hello! Due to speed up loading of a table of about 4 fields with 10 millions of record, are there some parameters to optimize? In same machine it takes about 15 minutes, in other machines about one hour. Parallel mode is better than direct mode? Other interesting things? Thank you in advance

Re: [GENERAL] BDR Selective Replication

2015-04-29 Thread Craig Ringer
On 29 April 2015 at 09:14, Jim Nasby jim.na...@bluetreble.com wrote: On 4/27/15 7:54 PM, Craig Ringer wrote: If 'default replication set' is the idea of here's what tables *should* be getting replicated regardless of whether that's happening or not, it'd be great if that was done

Re: [GENERAL] clearing of the transactions shown in pg_locks

2015-04-29 Thread Albe Laurenz
Mitu Verma wrote: I have killed the script, but still the query is showing in pg_stat and pg_locks. Please help me how to clear the pg_locks from the transaction it is already holding, if there is something in PostgreSQL which can clear the pg_stat and pg_locks? [...] fm_db_Server3=#

Re: [GENERAL] newsfeed type query

2015-04-29 Thread Ladislav Lenart
Hello. On 29.4.2015 00:26, Jonathan Vanasco wrote: I'm trying to upgrade some code that powers a newfeed type stream, and hoping someone can offer some insight on better ways to structure some parts of the query The part that has me stumped right now... There are several criteria

[GENERAL] New column modifier?

2015-04-29 Thread John McKown
I am wondering about useful something might be. So I hope ya'll don't mind me throwing out for feedback. I am fairly good with standard SQL, but not the more advanced DBA things such as TRIGGERs. I am reading good book, PostgreSQL Server Programming to increase my knowledge. The section that I'm

Re: [GENERAL] newsfeed type query

2015-04-29 Thread Jonathan Vanasco
On Apr 29, 2015, at 12:25 PM, Ladislav Lenart wrote: Could you please explain to me the error(s) in my reasoning? Let me just flip your list in reverse... and add in some elements (marked with a *): posting ts context p60 60 friend p55 55 friend* p54 54 friend* p50 50

Re: [GENERAL] newsfeed type query

2015-04-29 Thread Ladislav Lenart
Hello. On 29.4.2015 17:27, Jonathan Vanasco wrote: Thanks all! These point me in much better directions! Jim Nasby's approach to selecting an expression addressed some things (SELECT f.useraccount_id_b IS NOT NULL AS in_friends) Ladislav Lenart's usage of the CTE is also of a

Re: [GENERAL] PostgreSQL HA config recommendations

2015-04-29 Thread William Dunn
The streaming replication built into PostgreSQL would work fine for your use case, assuming that you are OK with having only one primary supporting writes and any slaves being read only as it currently (9.0-9.4) only supports a single master. This will put minimal load on your primary server and

[GENERAL] PostgreSQL HA config recommendations

2015-04-29 Thread Alex Gregory
Hello- I have been doing lots of reading and I really want to make sure that I get this HA architecture I am working on correct. I figured the best way would be to reach out to the community for advice. I am installing Cisco Jabber and want to use Postgres for the back end. The Postgres

Re: [GENERAL] PostgreSQL HA config recommendations

2015-04-29 Thread Joshua D. Drake
On 04/29/2015 10:53 AM, Alex Gregory wrote: Hello- I have been doing lots of reading and I really want to make sure that I get this HA architecture I am working on correct. I figured the best way would be to reach out to the community for advice. I am installing Cisco Jabber and want to

[GENERAL] Partition Help

2015-04-29 Thread akshunj
Hi, I am trying to setup partitions and as a test, I was able to follow the example in the Postgres docs using the date as a condition. Now I am trying to partition on a column with the data type character varying. I want to partition based on an invoice ID consisting on letters and numbers

Re: [GENERAL] Partition Help

2015-04-29 Thread Melvin Davidson
I think the problem is you need to specify NEW.invoice in all comparisons (don't quote the 'column')and always, Always, ALWAYS end each statement with a semicolon. IE: IF (NEW.invoice = 'I-1') THEN INSERT INTO myschema.mywork VALUES (NEW.*) ELSE IF (NEW.invoice = 'I-10001' AND NEW.invoice =

Re: [GENERAL] newsfeed type query

2015-04-29 Thread Ladislav Lenart
On 29.4.2015 18:54, Jonathan Vanasco wrote: On Apr 29, 2015, at 12:25 PM, Ladislav Lenart wrote: Could you please explain to me the error(s) in my reasoning? Let me just flip your list in reverse... and add in some elements (marked with a *): posting ts context p60 60 friend

[GENERAL]

2015-04-29 Thread Ramesh T
Hi all, as i mentioned above i have function called delete parts can i mention select deleteparts(); or c:\programfiles\9.4\pgagent\check.sql in this path check.sql contain to delete parts.i placed query select deleteparts(); when click run now job,it returns statistics failed and

Re: [GENERAL] New column modifier?

2015-04-29 Thread Joshua D. Drake
On 04/29/2015 10:03 AM, John McKown wrote: I am wondering about useful something might be. So I hope ya'll don't mind me throwing out for feedback. I am fairly good with standard SQL, but not the more advanced DBA things such as TRIGGERs. I am reading good book, PostgreSQL Server Programming to

Re: [GENERAL] New column modifier?

2015-04-29 Thread Melvin Davidson
What you suggest is a new column type, and that would be something that needs to be reviewed and implemented by the PostgreSQL developers. Since it is not a SQL standard, I doubt it has much chance of implementation. That being said, you should understand that all triggers require the need for an

Re: [GENERAL] New column modifier?

2015-04-29 Thread John McKown
On Wed, Apr 29, 2015 at 1:05 PM, Melvin Davidson melvin6...@gmail.com wrote: What you suggest is a new column type, and that would be something that needs to be reviewed and implemented by the PostgreSQL developers. Since it is not a SQL standard, I doubt it has much chance of implementation.

[GENERAL] PostgreSQL HA config recommendations

2015-04-29 Thread Alex Gregory
Hello- I have been doing lots of reading and I really want to make sure that I get this HA architecture I am working on correct. I figured the best way would be to reach out to the community for advice. I am installing Cisco Jabber and want to use Postgres for the back end. The Postgres

Re: [GENERAL]

2015-04-29 Thread Melvin Davidson
Sorry, my crystal ball is not working today. What is the PostgreSQL version? What is the O/S? What is the function definition? Where are you executing that command from? What is the error in the postgres log? Help us to help you. Please don't ask open ended questions without providing as much

Re: [GENERAL] Upgrading hot standbys

2015-04-29 Thread Bruce Momjian
On Wed, Apr 29, 2015 at 01:13:13PM +0200, Magnus Hagander wrote: On Wed, Apr 29, 2015 at 6:19 AM, Aaron Burnett aaron.burn...@us.dunnhumby.com wrote: Greetings, I'm in the process of upgrading PG 9.1.3 to 9.4.1 in the near future. I have several machines which each

Re: [GENERAL] PostgreSQL HA config recommendations

2015-04-29 Thread Alex Gregory
Thank for for the valuable feedback everyone. Much appreciated. As I look at the Jabber install doc they say to use 9.1.1(which I am hoping means 9.1.X (15) at least). I am afraid they may not be supported otherwise. The changes to replication logs relating to Replication Slots look

Re: [GENERAL] BDR Selective Replication

2015-04-29 Thread Jim Nasby
On 4/29/15 1:38 AM, Craig Ringer wrote: Perhaps... different replication systems probably use different methods to identify, so presumably there'd need to be some way to map a generic identifier into an appropriate identifier for whatever replication system you're using.

Re: [GENERAL] newsfeed type query

2015-04-29 Thread Jim Nasby
On 4/29/15 11:54 AM, Jonathan Vanasco wrote: IIRC, the best mix of performance and product that I've found is do something like this: SELECT * FROM ( SELECT a,b,c FROM table_a ORDER BY TIMESTAMP DESC LIMIT 1; UNION SELECT a,b,c FROM

Re: [GENERAL] New column modifier?

2015-04-29 Thread Jim Nasby
On 4/29/15 1:05 PM, Joshua D. Drake wrote: [ discussion about read-only columns ] See here GRANT { { SELECT | INSERT | UPDATE | REFERENCES } ( column_name [, ...] ) What I don't like about relying on GRANT is that the table owner gets to bypass all that, as does a superuser. So when I'm

Re: [GENERAL] Partition Help

2015-04-29 Thread Jim Nasby
On 4/29/15 10:05 AM, akshunj wrote: IF ('invoice' = 'I-1') THEN INSERT INTO myschema.mywork VALUES (NEW.*) ELSE IF ('invoice' = 'I-10001' AND = 'I-2' That's going to fall apart with invoice I-10. If you're going to go this route, depend on how IF ELSIF operates and don't try to

Re: [GENERAL] database split

2015-04-29 Thread Jim Nasby
On 4/29/15 6:04 PM, Dave Owens wrote: A. Text-processing a dump of the original database, filtering only the rows in which I am interested, while loading the dump into the new database. B. Copying the original database (CREATE DATABASE smalldb WITH TEMPLATE bigdb), then modifying the schema so

Re: [GENERAL] Pg_bulkload and speed

2015-04-29 Thread Jim Nasby
On 4/29/15 4:24 AM, Job wrote: Hello! Due to speed up loading of a table of about 4 fields with 10 millions of record, are there some parameters to optimize? In same machine it takes about 15 minutes, in other machines about one hour. There's a huge number of things that could account for

Re: [GENERAL] clearing of the transactions shown in pg_locks

2015-04-29 Thread Jim Nasby
On 4/29/15 12:32 AM, Mitu Verma wrote: This script first does the indexing then it deletes the entries from the table. Now issue is that this script is taking lot of time and has acquired some locks also. Nor sure if it is happening due to indexing or what. Mixing DDL with bulk DML operations

Re: [GENERAL] PostgreSQL HA config recommendations

2015-04-29 Thread Alex Gregory
What do you think of this article that I found detailing out how to install Postgres on CentOS, so that the proper fencing and STONITH is in place, using Pacemaker and Corosync?: https://github.com/smbambling/PGSQL_HA_CLUSTER/wiki/Building-A-Highly-Available-Multi-Node-PostgreSQL-Cluster

[GENERAL] database split

2015-04-29 Thread Dave Owens
Greetings, We are migrating a subset of our customers to a new set of servers. This requires that we migrate their data stored in postgresql (v9.1.15, Linux) to a new database. The new database happens to reside in the same cluster, but in the future this may not be the case. We are using a

Re: [GENERAL] PostgreSQL HA config recommendations

2015-04-29 Thread Jim Nasby
On 4/29/15 1:13 PM, Alex Gregory wrote: I was thinking that I could use Slony but then I read that it does not like WAN replication. I have also read about streaming replication native to Postgres but was not sure how that would work over the WAN. Bucardo seems better for Data Warehousing

Re: [GENERAL] newsfeed type query

2015-04-29 Thread Melvin Davidson
I see others have responded with suggestions to improve query performance, but one thing I noticed when you gave the data structure is there are no no primary keys defined for friends or posting, neither are there any indexes. Was that an omission? If not, then please note that PostgreSQL is a

Re: [GENERAL] newsfeed type query

2015-04-29 Thread Jonathan Vanasco
Thanks all! These point me in much better directions! Jim Nasby's approach to selecting an expression addressed some things (SELECT f.useraccount_id_b IS NOT NULL AS in_friends) Ladislav Lenart's usage of the CTE is also of a different format that I've used in the past. I think i'll be able

Re: [GENERAL] Pg_bulkload and speed

2015-04-29 Thread Takashi Ohnishi
Hello, Due to speed up loading of a table of about 4 fields with 10 millions of record, are there some parameters to optimize? In same machine it takes about 15 minutes, in other machines about one hour. Parallel mode is better than direct mode? Other interesting things? Yes, parallel