Re: [GENERAL] window function ordering not working as expected

2015-02-17 Thread Lonni J Friedman
On Tue, Feb 17, 2015 at 4:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: I'm interested in seeing: * the date for the most recent result * test name (identifier) * most recent result (decimal value) * the worst (lowest decimal value) test result from

[GENERAL] window function ordering not working as expected

2015-02-17 Thread Lonni J Friedman
Greetings, I have a postgresql-9.3.x database with a table with a variety of date stamped test results, some of which are stored in json format (natively in the database). I'm attempting to use some window functions to pull out specific data from the test results over a a time window, but part of

Re: [GENERAL] partitioned table + postgres_FDW not working in 9.3

2013-09-26 Thread Lonni J Friedman
shigeru.han...@gmail.com wrote: Hi Lonni, 2013/9/25 Lonni J Friedman netll...@gmail.com: The problem that I'm experiencing is if I attempt to perform an INSERT on the foreign nppsmoke table on cluster a, it fails claiming that the table partition which should hold the data in the INSERT does

Re: [GENERAL] partitioned table + postgres_FDW not working in 9.3

2013-09-26 Thread Lonni J Friedman
On Thu, Sep 26, 2013 at 8:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: Thanks for your reply. This sounds like a relatively simple workaround, so I'll give it a try. Is the search_path of the remote session that postgres_fdw forces considered

[GENERAL] pg_basebackup: ERROR: could not find any WAL files (9.3)

2013-09-26 Thread Lonni J Friedman
Greetings, I've recently pushed a new postgres-9.3 (Linux-x86_64/RHEL6) cluster into production, with one master, and two hot standby streaming replication slaves. Everything seems to be working ok, however roughly half of my pg_basebackup attempts are failing at the very end with the error:

[GENERAL] postgres FDW doesn't support sequences?

2013-09-25 Thread Lonni J Friedman
I've got two 9.3 clusters, with a postgres foreign data wrapper (FDW) setup to point from one cluster to the other. One of the (foreign) tables associated with the foreign server has a bigint sequence for its primary key, defined as: id | bigint | not null

Re: [GENERAL] postgres FDW doesn't support sequences?

2013-09-25 Thread Lonni J Friedman
On Wed, Sep 25, 2013 at 2:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: If I INSERT a new row into the local table (not the foreign table version), without specifying the 'id' column explicitly, it automatically is assigned the nextval in the sequence

[GENERAL] partitioned table + postgres_FDW not working in 9.3

2013-09-24 Thread Lonni J Friedman
Greetings, I've got two different 9.3 clusters setup, a b (on Linux if that matters). On cluster b, I have a table (nppsmoke) that is partitioned by date (month), which uses a function which is called by a trigger to manage INSERTS (exactly as documented in the official documentation for

Re: [GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-18 Thread Lonni J Friedman
On Wed, Sep 18, 2013 at 2:02 AM, Kevin Grittner kgri...@ymail.com wrote: Lonni J Friedman netll...@gmail.com wrote: top shows over 90% of the load is in sys space. vmstat output seems to suggest that its CPU bound (or bouncing back forth): Can you run `perf top` during an episode and see

Re: [GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-18 Thread Lonni J Friedman
On Wed, Sep 18, 2013 at 2:02 AM, Kevin Grittner kgri...@ymail.com wrote: Lonni J Friedman netll...@gmail.com wrote: top shows over 90% of the load is in sys space. vmstat output seems to suggest that its CPU bound (or bouncing back forth): Can you run `perf top` during an episode and see

[GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-17 Thread Lonni J Friedman
Greetings, I'm running a PostgreSQL 9.3.0 cluster (1 master with two streaming replication hot standby slaves) on RHEL6-x86_64. Yesterday I upgraded from 9.2.4 to 9.3.0, and since the upgrade I'm seeing a significant performance degradation. PostgreSQL simply feels slower. Nothing other than

Re: [GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-17 Thread Lonni J Friedman
On Tue, Sep 17, 2013 at 9:54 AM, Eduardo Morras emorr...@yahoo.es wrote: On Tue, 17 Sep 2013 09:19:29 -0700 Lonni J Friedman netll...@gmail.com wrote: Greetings, I'm running a PostgreSQL 9.3.0 cluster (1 master with two streaming replication hot standby slaves) on RHEL6-x86_64. Yesterday I

Re: [GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-17 Thread Lonni J Friedman
Thanks for your reply. Comments/answers inline below On Tue, Sep 17, 2013 at 11:28 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Sep 17, 2013 at 11:22 AM, Lonni J Friedman netll...@gmail.com wrote: c) What does logs say? The postgres server logs look perfectly normal, minus

Re: [GENERAL] upgrade from 9.2.x to 9.3 causes significant performance degradation

2013-09-17 Thread Lonni J Friedman
On Tue, Sep 17, 2013 at 3:47 PM, Andres Freund and...@2ndquadrant.com wrote: Hi, On 2013-09-17 09:19:29 -0700, Lonni J Friedman wrote: I'm running a PostgreSQL 9.3.0 cluster (1 master with two streaming replication hot standby slaves) on RHEL6-x86_64. Yesterday I upgraded from 9.2.4

Re: [GENERAL] WAL Replication Working but Not Working

2013-08-21 Thread Lonni J Friedman
The first thing to do is look at your server logs around the time when it stopped working. On Wed, Aug 21, 2013 at 7:08 AM, Joseph Marlin jmar...@saucontech.com wrote: We're having an issue with our warm standby server. About 9:30 last night, it stopped applying changes it received in WAL

Re: [GENERAL] Streaming Replication Randomly Locking Up

2013-08-15 Thread Lonni J Friedman
I've never seen this happen. Looks like you might be using 9.1? Are you up to date on all the 9.1.x releases? Do you have just 1 slave syncing from the master? Which OS are you using? Did you verify that there aren't any network problems between the slave master? Or hardware problems (like the

Re: [GENERAL] Streaming Replication Randomly Locking Up

2013-08-15 Thread Lonni J Friedman
again so if it is a network issue, the replication is just stopping after some hiccup instead of retrying and resuming when things are back up. Thanks! On Thu, Aug 15, 2013 at 11:32 AM, Lonni J Friedman netll...@gmail.com wrote: I've never seen this happen. Looks like you might be using

Re: [GENERAL] Streaming Replication Randomly Locking Up

2013-08-15 Thread Lonni J Friedman
= -1 #log_checkpoints = off #log_connections = off #log_disconnections = off #log_error_verbosity = default I'm going to have a look at the NICs to make sure there's no issue there. Thanks again for your help! On Thu, Aug 15, 2013 at 11:51 AM, Lonni J Friedman netll...@gmail.com wrote

[GENERAL] postgres FDW cost estimation options unrecognized in 9.3-beta1

2013-07-26 Thread Lonni J Friedman
Greetings, I have a postgresql-9.3-beta1 cluster setup (from the yum.postgresql.org RPMs), where I'm experimenting with the postgres FDW extension. The documentation ( http://www.postgresql.org/docs/9.3/static/postgres-fdw.html ) references three Cost Estimation Options which can be set for a

Re: [GENERAL] postgres FDW cost estimation options unrecognized in 9.3-beta1

2013-07-26 Thread Lonni J Friedman
On Fri, Jul 26, 2013 at 3:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: nightly=# ALTER SERVER cuda_db10 OPTIONS (SET use_remote_estimate 'true') ; ERROR: option use_remote_estimate not found Am I doing something wrong, or is this a bug

[GENERAL] upgrading from 9.3-beta1 to 9.3-beta2 requires dump reload?

2013-07-24 Thread Lonni J Friedman
Greetings, I just got around to upgrading from 9.3-beta1 to 9.3-beta2, and was surprised to see that the server was refusing to start. In the log, I'm seeing: 2013-07-24 13:41:47 PDT [7083]: [1-1] db=,user= FATAL: database files are incompatible with server 2013-07-24 13:41:47 PDT [7083]: [2-1]

Re: [GENERAL] upgrading from 9.3-beta1 to 9.3-beta2 requires dump reload?

2013-07-24 Thread Lonni J Friedman
On Wed, Jul 24, 2013 at 2:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Lonni J Friedman escribió: I'm using the RPMs from yum.postgresql.org on RHEL6. Is this expected, intentional behavior? Do I really need to dump reload to upgrade between

Re: [GENERAL] Standby stopped working after PANIC: WAL contains references to invalid pages

2013-06-22 Thread Lonni J Friedman
Looks like some kind of data corruption. Question is whether it came from the master, or was created by the standby. If you re-seed the standby with a full (base) backup, does the problem go away? On Sat, Jun 22, 2013 at 12:43 PM, Dan Kogan d...@iqtell.com wrote: Hello, Today our standby

Re: [GENERAL] Standby stopped working after PANIC: WAL contains references to invalid pages

2013-06-22 Thread Lonni J Friedman
and has been working for about 2 hours. The file in the error message was an index. We rebuilt it just in case. Is there any way to debug the issue at this point? -Original Message- From: Lonni J Friedman [mailto:netll...@gmail.com] Sent: Saturday, June 22, 2013 4:11 PM To: Dan

[GENERAL] 9.3-beta postgres-fdw COPY error

2013-06-21 Thread Lonni J Friedman
Greetings, I'm trying to test out the new postgres-fdw support in postgresql-9.3 (beta) in preparation for an upgrade from 9.2 later this year. So far, everything is working ok, however one problem I'm encountering is with the COPY command. When I run it against a foreign table (which is also in

Re: [GENERAL] 9.3-beta postgres-fdw COPY error

2013-06-21 Thread Lonni J Friedman
AM, Lonni J Friedman wrote: Greetings, I'm trying to test out the new postgres-fdw support in postgresql-9.3 (beta) in preparation for an upgrade from 9.2 later this year. So far, everything is working ok, however one problem I'm encountering is with the COPY command. When I run it against

[GENERAL] how to reference variables in pgbench custom scripts?

2013-06-18 Thread Lonni J Friedman
I'm attempting to write a custom pgbench script (called via the -f option), with a variable set at the top with: \setrandom aid 100 50875000 However, I can't quite figure out how to reference the new aid variable. The documentation simply states that a variable is referenced with a colon in

Re: [GENERAL] pg_upgrade link mode

2013-05-16 Thread Lonni J Friedman
On Thu, May 16, 2013 at 11:03 AM, Igor Neyman iney...@perceptron.com wrote: From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of AI Rumman Sent: Thursday, May 16, 2013 1:56 PM To: Fabio Rueda Carrascosa Cc: pgsql-general Subject: Re: [GENERAL]

Re: [GENERAL] pg_basebackup, requested WAL has already been removed

2013-05-10 Thread Lonni J Friedman
Its definitely not a bug. You need to set/increase wal_keep_segments to a value that ensures that they aren't recycled faster than the time required to complete the base backup (plus some buffer). On Fri, May 10, 2013 at 9:48 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: Hi, I've recently

Re: [GENERAL] pg_basebackup, requested WAL has already been removed

2013-05-10 Thread Lonni J Friedman
kopo...@ast.cam.ac.uk wrote: On Fri, 10 May 2013, Lonni J Friedman wrote: Its definitely not a bug. You need to set/increase wal_keep_segments to a value that ensures that they aren't recycled faster than the time required to complete the base backup (plus some buffer). But I thought

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Lonni J Friedman
On Fri, May 10, 2013 at 10:20 AM, Merlin Moncure mmonc...@gmail.com wrote: On Fri, May 10, 2013 at 12:03 PM, David Boreham david_l...@boreham.org wrote: On 5/10/2013 10:21 AM, Merlin Moncure wrote: As it turns out the list of flash drives are suitable for database use is surprisingly small.

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-10 Thread Lonni J Friedman
On Fri, May 10, 2013 at 11:23 AM, Steven Schlansker ste...@likeness.com wrote: On May 10, 2013, at 7:14 AM, Matt Brock m...@mattbrock.co.uk wrote: Hello. We're intending to deploy PostgreSQL on Linux with SSD drives which would be in a RAID 1 configuration with Hardware RAID. My first

Re: [GENERAL] Replication terminated due to PANIC

2013-04-25 Thread Lonni J Friedman
If its really index corruption, then you should be able to fix it by reindexing. However, that doesn't explain what caused the corruption. Perhaps your hardware is bad in some way? On Wed, Apr 24, 2013 at 10:46 PM, Adarsh Sharma eddy.ada...@gmail.com wrote: Thanks Sergey for such a quick

Re: [GENERAL] corrupted item pointer in streaming based replication

2013-04-03 Thread Lonni J Friedman
You should figure out what base/16384/114846.39 corresponds to inside the database. If you're super lucky its something unimportant and/or something that can be recreated easily (like an index). If its something important, then you're only option is to try to drop the object and restore it from

Re: [GENERAL] Streaming replication slave crash

2013-03-29 Thread Lonni J Friedman
Looks like you've got some form of coruption: page 1441792 of relation base/63229/63370 does not exist The question is whether it was corrupted on the master and then replicated to the slave, or if it was corrupted on the slave. I'd guess that the pg_dump tried to read from that page and

Re: [GENERAL] replication behind high lag

2013-03-25 Thread Lonni J Friedman
On Mon, Mar 25, 2013 at 12:37 PM, AI Rumman rumman...@gmail.com wrote: Hi, I have two 9.2 databases running with hot_standby replication. Today when I was checking, I found that replication has not been working since Mar 1st. There was a large database restored in master on that day and I

Re: [GENERAL] replication behind high lag

2013-03-25 Thread Lonni J Friedman
On Mon, Mar 25, 2013 at 12:43 PM, AI Rumman rumman...@gmail.com wrote: On Mon, Mar 25, 2013 at 3:40 PM, Lonni J Friedman netll...@gmail.com wrote: On Mon, Mar 25, 2013 at 12:37 PM, AI Rumman rumman...@gmail.com wrote: Hi, I have two 9.2 databases running with hot_standby replication

Re: [GENERAL] replication behind high lag

2013-03-25 Thread Lonni J Friedman
On Mon, Mar 25, 2013 at 12:55 PM, AI Rumman rumman...@gmail.com wrote: On Mon, Mar 25, 2013 at 3:52 PM, Lonni J Friedman netll...@gmail.com wrote: On Mon, Mar 25, 2013 at 12:43 PM, AI Rumman rumman...@gmail.com wrote: On Mon, Mar 25, 2013 at 3:40 PM, Lonni J Friedman netll...@gmail.com

Re: [GENERAL] replication behind high lag

2013-03-25 Thread Lonni J Friedman
On Mon, Mar 25, 2013 at 1:23 PM, AI Rumman rumman...@gmail.com wrote: On Mon, Mar 25, 2013 at 4:03 PM, AI Rumman rumman...@gmail.com wrote: On Mon, Mar 25, 2013 at 4:00 PM, Lonni J Friedman netll...@gmail.com wrote: On Mon, Mar 25, 2013 at 12:55 PM, AI Rumman rumman...@gmail.com wrote

Re: [GENERAL] UNLOGGED TEMPORARY tables?

2013-03-25 Thread Lonni J Friedman
I'm pretty sure that unlogged tables and temp tables are two separate distinct features, with no overlap in functionality. It would be nice if it was possible to create an unlogged temp table. On Sun, Mar 24, 2013 at 1:32 PM, aasat satri...@veranet.pl wrote: I was tested write speed to

Re: [GENERAL] UNLOGGED TEMPORARY tables?

2013-03-25 Thread Lonni J Friedman
On Mon, Mar 25, 2013 at 4:49 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Mar 26, 2013 at 8:26 AM, Lonni J Friedman netll...@gmail.com wrote: I'm pretty sure that unlogged tables and temp tables are two separate distinct features, with no overlap in functionality. It would

Re: [GENERAL] Replication stopped on 9.0.2 after making change to conf file

2013-03-09 Thread Lonni J Friedman
It sounds like all you did was setup the slave from scratch with a fresh base backup, without understanding or debugging what caused everything to break. Clearly whatever was wrong on March 5 is still wrong, and nothing has been fixed. The first step in debugging this problem is to look at

Re: [GENERAL] postgres 9.0.2 replicated database is crashing

2013-03-09 Thread Lonni J Friedman
That process merely sets up a new server, it doesn't start streaming, unless the server has been configured correctly. You state that the slave crashed after two hours. How did you make this determination? All you seem to be doing is setting up the slave from scratch repeatedly, and assuming

Re: [GENERAL] postgres 9.0.2 replicated database is crashing

2013-03-09 Thread Lonni J Friedman
On Sat, Mar 9, 2013 at 1:51 PM, akp geek akpg...@gmail.com wrote: thank you. As you mentioned, I understood that I am starting the streaming scratch which is not what I wanted to do. Here is what I am planning to . Our replication process was down since March5th. 1. Is it Ok to get all

Re: [GENERAL] postgres 9.0.2 replicated database is crashing

2013-03-09 Thread Lonni J Friedman
looking for the most straightforward path I'd recommend going to 9.0.12. Also be sure to read the release notes first. We use GIST indexes quite a bit. and we gis also I recently compiled postgres 9.2 .. Regards On Sat, Mar 9, 2013 at 5:09 PM, Lonni J Friedman netll...@gmail.com wrote

Re: [GENERAL] broke postgres, how to fix??

2013-02-26 Thread Lonni J Friedman
Did you shut down the 'old' postgres before copying these files? Did you (re)configure the 'new' postgres to set its $PGDATA directory to the location of the 'new' files? On Fri, Feb 22, 2013 at 3:46 PM, JD Wong jdmsw...@gmail.com wrote: I tried copying postgres over to a new directory. it was

Re: [GENERAL] broke postgres, how to fix??

2013-02-26 Thread Lonni J Friedman
On Tue, Feb 26, 2013 at 4:02 PM, JD Wong jdmsw...@gmail.com wrote: Hi Adrian, yes I completely copied the config-file and data directories over. Lonnie, I don't remember. I might not have shut down the old postgres, yes I set PGDATA accordingly. That's guaranteed to break everything badly.

Re: [GENERAL] broke postgres, how to fix??

2013-02-26 Thread Lonni J Friedman
does postgres know about this? http://www.postgresql.org/docs/9.2/static/backup-file.html Thanks, -JD On Tue, Feb 26, 2013 at 7:04 PM, Lonni J Friedman netll...@gmail.com wrote: On Tue, Feb 26, 2013 at 4:02 PM, JD Wong jdmsw...@gmail.com wrote: Hi Adrian, yes I completely copied

[GENERAL] special procedure required when running pg_basebackup from a standby?

2013-01-13 Thread Lonni J Friedman
Greetings, I'm running postgres-9.2.2 in a Linux-x86_64 cluster with 1 master and several hot standby servers. Since upgrading to 9.2.2 from 9.1.x a few months ago, I switched from generating a base backup on the master, to generating it on a dedicated slave/standby (to reduce the load on the

[GENERAL] data corruption when using base backups generated from hot standby

2013-01-10 Thread Lonni J Friedman
Greetings, I'm running postgres-9.2.2 in a Linux-x86_64 cluster with 1 master and several hot standby servers. Since upgrading to 9.2.2 from 9.1.x a few months ago, I switched from generating a base backup on the master, to generating it on a dedicated slave/standby (to reduce the load on the

Re: [GENERAL] pgpool2 load balancing not working

2013-01-04 Thread Lonni J Friedman
On Fri, Jan 4, 2013 at 3:42 PM, Greg Donald gdon...@gmail.com wrote: Sorry if this is the wrong list, but I've been stuck for a couple days now. I tried pgpool-general but that list appears to not like me. I'm not getting any posts and my post hasn't shown up in the archives. Specifically

Re: [GENERAL] Postgresql logfilename and times in GMT - not EST

2012-12-04 Thread Lonni J Friedman
On Tue, Dec 4, 2012 at 1:59 PM, Bryan Montgomery mo...@english.net wrote: We have a test 9.2.0 db running on openSuse 12.2. When I select now() I get the correct timezone and date back (-5 hours). When I do date at the os prompt, I get the right timezone back. I changed postgres.conf to have

Re: [GENERAL] Postgresql logfilename and times in GMT - not EST

2012-12-04 Thread Lonni J Friedman
On Tue, Dec 4, 2012 at 2:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: On Tue, Dec 4, 2012 at 1:59 PM, Bryan Montgomery mo...@english.net wrote: I changed postgres.conf to have timezone = 'EST' and restarted postgres. However the log file is still 5

Re: [GENERAL] Table with million rows - and PostgreSQL 9.1 is not using the index

2012-12-04 Thread Lonni J Friedman
I'm no expert on this, but it will likely be more helpful to others if you include the table description with all the indices. On Tue, Dec 4, 2012 at 8:44 PM, Edson Richter edsonrich...@hotmail.com wrote: I've a table with 110 rows, with streets. I'm making a partial search using zip code,

Re: [GENERAL] Quick estimate of num of rows table size

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 2:02 PM, Thalis Kalfigkopoulos tkalf...@gmail.com wrote: Hi all, I read somewhere that the following query gives a quick estimate of the # of rows in a table regardless of the table's size (which would matter in a simple SELECT count(*)?): SELECT (CASE WHEN reltuples

Re: [GENERAL] Quick estimate of num of rows table size

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 3:56 PM, Thalis Kalfigkopoulos tkalf...@gmail.com wrote: On Mon, Nov 5, 2012 at 7:14 PM, Lonni J Friedman netll...@gmail.com wrote: On Mon, Nov 5, 2012 at 2:02 PM, Thalis Kalfigkopoulos tkalf...@gmail.com wrote: Hi all, I read somewhere that the following query

Re: [GENERAL] Hot Standby Not So Hot Anymore

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 7:40 PM, Ian Harding harding@gmail.com wrote: I had a 9.0.8 hot standby setup, one master, two slaves, working great. Then, I tried to re-initialize by making a base backup, the way I've done it many times before, but for some reason I can't get the standby to accept

Re: [GENERAL] Hot Standby Not So Hot Anymore

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 7:49 PM, Ian Harding harding@gmail.com wrote: On Mon, Nov 5, 2012 at 7:46 PM, Lonni J Friedman netll...@gmail.com wrote: On Mon, Nov 5, 2012 at 7:40 PM, Ian Harding harding@gmail.com wrote: I had a 9.0.8 hot standby setup, one master, two slaves, working great

Re: [GENERAL] Hot Standby Not So Hot Anymore

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 8:13 PM, Ian Harding harding@gmail.com wrote: On Mon, Nov 5, 2012 at 7:57 PM, Lonni J Friedman netll...@gmail.com wrote: On Mon, Nov 5, 2012 at 7:49 PM, Ian Harding harding@gmail.com wrote: On Mon, Nov 5, 2012 at 7:46 PM, Lonni J Friedman netll

Re: [GENERAL] Hot Standby Not So Hot Anymore

2012-11-05 Thread Lonni J Friedman
On Mon, Nov 5, 2012 at 8:31 PM, Ian Harding harding@gmail.com wrote: On Mon, Nov 5, 2012 at 8:15 PM, Lonni J Friedman netll...@gmail.com wrote: On Mon, Nov 5, 2012 at 8:13 PM, Ian Harding harding@gmail.com wrote: On Mon, Nov 5, 2012 at 7:57 PM, Lonni J Friedman netll

Re: [GENERAL] dropdb breaks replication?

2012-10-31 Thread Lonni J Friedman
On Wed, Oct 31, 2012 at 10:32 AM, Edson Richter edsonrich...@hotmail.com wrote: I've two PostgreSQL 9.1.6 running on Linux CentOS 5.8 64bit. They are replicated asynchronously. Yesterday, I've dropped a database of 20Gb, and then replication has broken, requiring me to manually synchronize

Re: [GENERAL] dropdb breaks replication?

2012-10-31 Thread Lonni J Friedman
On Wed, Oct 31, 2012 at 11:01 AM, Edson Richter edsonrich...@hotmail.com wrote: Em 31/10/2012 15:39, Lonni J Friedman escreveu: On Wed, Oct 31, 2012 at 10:32 AM, Edson Richter edsonrich...@hotmail.com wrote: I've two PostgreSQL 9.1.6 running on Linux CentOS 5.8 64bit. They are replicated

Re: [GENERAL] 9.1 to 9.2 requires a dump/reload?

2012-10-22 Thread Lonni J Friedman
pg_upgrade has worked fine for several releases. I believe that the only time when pg_upgrade isn't a viable option is for some types of GIST indices. On Mon, Oct 22, 2012 at 2:55 PM, Nikolas Everett nik9...@gmail.com wrote: I was just looking at

Re: [GENERAL] Strategies/Best Practises Handling Large Tables

2012-10-12 Thread Lonni J Friedman
On Fri, Oct 12, 2012 at 7:44 AM, Chitra Creta chitracr...@gmail.com wrote: Hi, I currently have a table that is growing very quickly - i.e 7 million records in 5 days. This table acts as a placeholder for statistics, and hence the records are merely inserted and never updated or deleted.

Re: [GENERAL] strange hot_standby behaviour

2012-10-01 Thread Lonni J Friedman
On Mon, Oct 1, 2012 at 7:28 AM, pfote pf...@ypsilon.net wrote: Hi, I had a very strange effect on the weekend that smells like a bug, so i'd like so share it. Setup: machine A: 16 CPU Cores (modern), 128GB RAM, nice 6-drive SAS Raid-10 machines B, C: 8 Cores (substantially older than A),

Re: [GENERAL] initial sync of multiple streaming slaves simultaneously

2012-09-19 Thread Lonni J Friedman
On Wed, Sep 19, 2012 at 8:59 AM, Mike Roest mike.ro...@replicon.com wrote: Hey Everyone, We currently have a 9.1.5 postgres cluster running using streaming replication. We have 3 nodes right now 2 - local that are setup with pacemaker for a HA master/slave set failover cluster 1 -

Re: [GENERAL] initial sync of multiple streaming slaves simultaneously

2012-09-19 Thread Lonni J Friedman
Just curious, is there a reason why you can't use pg_basebackup ? On Wed, Sep 19, 2012 at 12:27 PM, Mike Roest mike.ro...@replicon.com wrote: Is there any hidden issue with this that we haven't seen. Or does anyone have suggestions as to an alternate procedure that will allow 2 slaves to

Re: [GENERAL] File system level backup

2012-07-26 Thread Lonni J Friedman
On Thu, Jul 26, 2012 at 3:39 AM, Manoj Agarwal m...@ockham.be wrote: Hi, I have two virtual machines with two different versions of Postgresql. One machine contains Postgres 7.4.19 and another has Postgres 8.4.3. I also have other instances of these two virtual machines. I need to

Re: [GENERAL] insert binary data into a table column with psql

2012-07-24 Thread Lonni J Friedman
On Tue, Jul 24, 2012 at 2:22 PM, John R Pierce pie...@hogranch.com wrote: On 07/24/12 1:28 PM, jkells wrote: from psql I have tried several ways including creating a function to read a file without any success but basically I want to do something like the following from a bash shell psql

Re: [GENERAL] insert binary data into a table column with psql

2012-07-24 Thread Lonni J Friedman
On Tue, Jul 24, 2012 at 7:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: jtkells jtke...@verizon.net writes: Thanks much for your reply, that does the trick quite nicely. But, I just came to the realization that this only works if your are running the client and the file both resides on the

Re: [GENERAL] big database resulting in small dump

2012-07-20 Thread Lonni J Friedman
On Fri, Jul 20, 2012 at 11:05 AM, Ilya Ivanov f...@ngs.ru wrote: I have a 8.4 database (installed on ubuntu 10.04 x86_64). It holds Zabbix database. The database on disk takes 10Gb. SQL dump takes only 2Gb. I've gone through http://archives.postgresql.org/pgsql-general/2008-08/msg00316.php and

Re: [GENERAL] big database resulting in small dump

2012-07-20 Thread Lonni J Friedman
On Fri, Jul 20, 2012 at 11:23 AM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: On Fri, Jul 20, 2012 at 11:05 AM, Ilya Ivanov f...@ngs.ru wrote: I have a 8.4 database (installed on ubuntu 10.04 x86_64). It holds Zabbix database. The database on disk takes 10Gb

Re: [GENERAL] efficiency of wildcards at both ends

2012-06-20 Thread Lonni J Friedman
On Wed, Jun 20, 2012 at 10:10 AM, Sam Z J sammyjiang...@gmail.com wrote: Hi all I'm curious how is wildcards at both ends implemented, e.g. LIKE '%str%' How efficient is it if that's the only search criteria against a large table? how much does indexing the column help and roughly how much

Re: [GENERAL] pg_basebackup blocking all queries

2012-06-06 Thread Lonni J Friedman
on any query (read or write) being horrible (seconds to minutes). As soon as the basebackup completes, perf returns to normal (and the load drops back down to 1.00 or less). How can I debug what's wrong? On Tue, May 22, 2012 at 3:20 PM, Lonni J Friedman netll...@gmail.com wrote: Thanks for your reply

Re: [GENERAL] autovacuum running for a long time on a new table with 1 row

2012-06-01 Thread Lonni J Friedman
On Fri, Jun 1, 2012 at 10:34 AM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: Running 9.1.3 on Linux-x86_64.  I'm seeing autovacuum running for the past 6 hours on a newly created table that only has 1 row of data in it.  This table did exist previously

Re: [GENERAL] autovacuum running for a long time on a new table with 1 row

2012-06-01 Thread Lonni J Friedman
On Fri, Jun 1, 2012 at 10:54 AM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: On Fri, Jun 1, 2012 at 10:34 AM, Tom Lane t...@sss.pgh.pa.us wrote: This seems to have been noticed and fixed in HEAD: http://git.postgresql.org/gitweb/?p=postgresql.gita=commitdiffh

[GENERAL] autovacuum running for a long time on a new table with 1 row

2012-05-31 Thread Lonni J Friedman
Running 9.1.3 on Linux-x86_64. I'm seeing autovacuum running for the past 6 hours on a newly created table that only has 1 row of data in it. This table did exist previously, but was dropped recreated. I'm not sure if that might explain this behavior. When I strace the autovacuum process, I

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 - 9.1

2012-05-25 Thread Lonni J Friedman
On Thu, May 24, 2012 at 12:57 PM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: On Thu, May 24, 2012 at 12:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: Can you correlate the performance hit with any specific part of autovacuum? In particular, I'm wondering

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 - 9.1

2012-05-24 Thread Lonni J Friedman
On Wed, May 23, 2012 at 2:45 PM, Gavin Flower gavinflo...@archidevsys.co.nz wrote: On 24/05/12 08:18, Lonni J Friedman wrote: On Wed, May 23, 2012 at 12:36 PM, Gavin Flower gavinflo...@archidevsys.co.nz wrote: On 24/05/12 05:09, Lonni J Friedman wrote: On Wed, May 23, 2012 at 9:37 AM, Tom

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 - 9.1

2012-05-24 Thread Lonni J Friedman
On Thu, May 24, 2012 at 12:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: No, not lots of subqueries or ORDERing, and most queries only touch a single table.  However, I'm honestly not sure that I'm following where you're going with this.   The problem

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 - 9.1

2012-05-23 Thread Lonni J Friedman
Thanks for your reply. On Tue, May 22, 2012 at 7:19 PM, Andy Colson a...@squeakycode.net wrote:  On Mon, May 21, 2012 at 2:05 PM, Lonni J Friedmannetll...@gmail.com  wrote: Greetings, When I got in this morning, I found an autovacuum process that had been running since just before the load

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 - 9.1

2012-05-23 Thread Lonni J Friedman
On Wed, May 23, 2012 at 9:37 AM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: After banging my head on the wall for  a long time, I happened to notice that khugepaged was consuming 100% CPU every time autovacuum was running.  I did: echo madvise /sys/kernel

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 - 9.1

2012-05-23 Thread Lonni J Friedman
On Wed, May 23, 2012 at 12:36 PM, Gavin Flower gavinflo...@archidevsys.co.nz wrote: On 24/05/12 05:09, Lonni J Friedman wrote: On Wed, May 23, 2012 at 9:37 AM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: After banging my head on the wall for  a long time

Re: [GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 - 9.1

2012-05-23 Thread Lonni J Friedman
On Wed, May 23, 2012 at 3:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Gavin Flower gavinflo...@archidevsys.co.nz writes: 16 core Xeon X5550 2.67GHz 128GB RAM $PGDATA sits on a RAID5 array comprised of 3 SATA disks.  Its Linux's md software RAID. How does this compare to your other machines

[GENERAL] Re: significant performance hit whenever autovacuum runs after upgrading from 9.0 - 9.1

2012-05-22 Thread Lonni J Friedman
No one has any ideas or suggestions, or even questions? If someone needs more information, I'd be happy to provide it. This problem is absolutely killing me. On Mon, May 21, 2012 at 2:05 PM, Lonni J Friedman netll...@gmail.com wrote: Greetings, I have a 4 server postgresql-9.1.3 cluster (one

[GENERAL] pg_basebackup blocking all queries

2012-05-22 Thread Lonni J Friedman
Greetings, I have a 4 server postgresql-9.1.3 cluster (one master doing streaming replication to 3 hot standby servers). All of them are running Fedora-16-x86_64. Last Friday I upgraded the entire cluster from Fedora-15 with postgresql-9.0.6 to Fedora-16 with postgresql-9.1.3. I'm finding that

Re: [GENERAL] pg_basebackup blocking all queries

2012-05-22 Thread Lonni J Friedman
scott.marl...@gmail.com wrote: Do the queries here help? http://wiki.postgresql.org/wiki/Lock_Monitoring On Tue, May 22, 2012 at 12:42 PM, Lonni J Friedman netll...@gmail.com wrote: Greetings, I have a 4 server postgresql-9.1.3 cluster (one master doing streaming replication to 3 hot standby

[GENERAL] significant performance hit whenever autovacuum runs after upgrading from 9.0 - 9.1

2012-05-21 Thread Lonni J Friedman
Greetings, I have a 4 server postgresql-9.1.3 cluster (one master doing streaming replication to 3 hot standby servers). All of them are running Fedora-16-x86_64. Last Friday I upgraded the entire cluster from Fedora-15 with postgresql-9.0.6 to Fedora-16 with postgresql-9.1.3. I made no changes

[GENERAL] problems after restoring from a pg_basebackup

2012-04-27 Thread Lonni J Friedman
Greetings, I'm running postgresql-9.1.3 on a Linux-x86_64 (Fedora16, if it matters) system. I noticed the existence of pg_basebackup starting in 9.1, and figured I'd try it out and see if it would simplify our backup management processes. $ pg_basebackup -P -v -D /tmp/backup -x -Ft -z -U

Re: [GENERAL] pg_basebackup issues

2012-04-24 Thread Lonni J Friedman
On Fri, Apr 20, 2012 at 12:31 PM, Magnus Hagander mag...@hagander.net wrote: On Fri, Apr 20, 2012 at 19:51, Lonni J Friedman netll...@gmail.com wrote: Anyway, lesson learned, I need to either invoke pg_basebackup as the same user that runs the database (or is specified with the -U parameter

[GENERAL] pg_basebackup issues

2012-04-20 Thread Lonni J Friedman
Greetings, I'm running postgresql-9.1.3 on a Linux-x86_64 (Fedora16, if it matters) system. I noticed the existence of pg_basebackup starting in 9.1, and figured I'd try it out and see if it would simplify our backup management processes. I setup a test system (same OS postgresql version as

Re: [GENERAL] pg_upgrade + streaming replication ?

2012-03-20 Thread Lonni J Friedman
On Tue, Mar 20, 2012 at 11:46 AM, Bruce Momjian br...@momjian.us wrote: On Mon, Mar 19, 2012 at 03:07:02PM -0700, Jeff Davis wrote: On Mon, 2012-03-19 at 15:30 -0400, Bruce Momjian wrote: On Thu, Mar 01, 2012 at 02:01:31PM -0800, Lonni J Friedman wrote: I've got a 3 node cluster (1 master/2

Re: [GENERAL] pg_upgrade + streaming replication ?

2012-03-19 Thread Lonni J Friedman
On Mon, Mar 19, 2012 at 12:30 PM, Bruce Momjian br...@momjian.us wrote: On Thu, Mar 01, 2012 at 02:01:31PM -0800, Lonni J Friedman wrote: I've got a 3 node cluster (1 master/2 slaves) running 9.0.x with streaming replication.  I'm in the planning stages of upgrading to 9.1.x, and am looking

Re: [GENERAL] how to measure wal_buffer usage

2012-03-16 Thread Lonni J Friedman
On Fri, Mar 16, 2012 at 2:45 AM, Albe Laurenz laurenz.a...@wien.gv.at wrote: Lonni J Friedman wrote: After reading this interesting article on shared_buffers and wal_buffers: http://rhaas.blogspot.com/2012/03/tuning-sharedbuffers-and-walbuffers.html it got me wondering if my settings were

[GENERAL] how to measure wal_buffer usage

2012-03-15 Thread Lonni J Friedman
After reading this interesting article on shared_buffers and wal_buffers: http://rhaas.blogspot.com/2012/03/tuning-sharedbuffers-and-walbuffers.html it got me wondering if my settings were ideal. Is there some way to measure wal_buffer usage in real time, so that I could simply monitor it for

[GENERAL] pg_upgrade + streaming replication ?

2012-03-01 Thread Lonni J Friedman
I've got a 3 node cluster (1 master/2 slaves) running 9.0.x with streaming replication. I'm in the planning stages of upgrading to 9.1.x, and am looking into the most efficient way to do the upgrade with the goal of minimizing downtime risk. After googling, the only discussion that I've found

[GENERAL] returning rows from an implicit JOIN where results either exist in both tables OR only one table

2011-12-01 Thread Lonni J Friedman
Greetings, I've got a PostgreSQL-9.0.x database that manages an automated testing environment. There are a bunch of tables that contain assorted static data (OS versions, test names, etc) named 'buildlist' 'osversmap'. However, there are also two tables which contain data which changes often.

Re: [GENERAL] returning rows from an implicit JOIN where results either exist in both tables OR only one table

2011-12-01 Thread Lonni J Friedman
On Thu, Dec 1, 2011 at 1:57 PM, David Johnston pol...@yahoo.com wrote: -Original Message- From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of Lonni J Friedman Sent: Thursday, December 01, 2011 4:13 PM To: pgsql-general Subject: [GENERAL

[GENERAL] autovacuum stuck on a table for 18+ hours, consuming lots of CPU time

2011-11-22 Thread Lonni J Friedman
Greetings, I'm running PostgreSQL-9.0.4 on a Linux-x86_64 cluster with 1 master, and two streaming replication slaves. Since late yesterday, the load on the server has been noticably higher (5.00+) than normal (generally under 1.00). I investigated, and found that for the past ~18 hours, there's

Re: [GENERAL] autovacuum stuck on a table for 18+ hours, consuming lots of CPU time

2011-11-22 Thread Lonni J Friedman
On Tue, Nov 22, 2011 at 6:57 PM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: When I strace PID 30188, I see tons of this scrolling past quickly, but I'm not really sure what it means beyond a 'Timeout' not looking good: select(0, NULL, NULL, NULL, {0, 32000

  1   2   3   >