Re: [GENERAL] Disk Encryption in Production

2014-03-25 Thread Zenaan Harkness
On 3/26/14, Carlos Espejo wrote: > Anybody running their PostgreSQL server from a ecryptfs container? What are > the common production setups out there? What are the drawbacks that people > have experienced with their solution? I ran a couple web servers off Full Disk Encryption (FDE) installs, a

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Adrian Klaver
On 03/25/2014 04:52 PM, Steven Schlansker wrote: Some more questions, what happens when things begin to dawn on me:) You said the disk filled up entirely with log files yet currently the number(size) of logs is growing. It’s holding stable now. I tried to vacuum up to clean some space wh

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Jeff Janes
On Tuesday, March 25, 2014, Steven Schlansker wrote: > Hi everyone, > > I have a Postgres 9.3.3 database machine. Due to some intelligent work on > the part of someone who shall remain nameless, the WAL archive command > included a '> /dev/null 2>&1' which masked archive failures until the disk

Re: [GENERAL] Failure upgrading PG 9.2 to 9.3

2014-03-25 Thread Adrian Klaver
On 03/25/2014 05:14 PM, Sam Saffron wrote: 9.2 is the problem instance, 9.3 is clean, I am able to do many upgrades without issues with the same script (which spawns a clean 9.3 instance and then pg_upgrades to it.) Alright that makes sense, though I am still unclear about the below from your

Re: [GENERAL] Failure upgrading PG 9.2 to 9.3

2014-03-25 Thread Sam Saffron
9.2 is the problem instance, 9.3 is clean, I am able to do many upgrades without issues with the same script (which spawns a clean 9.3 instance and then pg_upgrades to it.) On Wed, Mar 26, 2014 at 11:13 AM, Adrian Klaver wrote: > On 03/25/2014 05:03 PM, Sam Saffron wrote: >> >> Yes Adrian, >> >>

Re: [GENERAL] Failure upgrading PG 9.2 to 9.3

2014-03-25 Thread Adrian Klaver
On 03/25/2014 05:03 PM, Sam Saffron wrote: Yes Adrian, That is correct (upgraded 9.2 to 9.3 via pg_dump), more than happy to provide pg devs with the actual db if needed. Pretty sure the target db is good, especially since we just dumped a single db (did not do a dump_all) Well it has more to

Re: [GENERAL] Failure upgrading PG 9.2 to 9.3

2014-03-25 Thread Sam Saffron
Yes Adrian, That is correct (upgraded 9.2 to 9.3 via pg_dump), more than happy to provide pg devs with the actual db if needed. Pretty sure the target db is good, especially since we just dumped a single db (did not do a dump_all) On Wed, Mar 26, 2014 at 10:58 AM, Adrian Klaver wrote: > On 03/

Re: [GENERAL] Failure upgrading PG 9.2 to 9.3

2014-03-25 Thread Adrian Klaver
On 03/25/2014 04:32 PM, Sam Saffron wrote: Thanks heaps Tom, I can confirm corrupt db upgrades fine with pg_dump. Was wondering if there are any plans to add a --no-validate to pg_upgrade, since the crash seems only to happen during validation. Hmm, so I am still unclear on this. The 'corrupt'

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Steven Schlansker
On Mar 25, 2014, at 4:45 PM, Adrian Klaver wrote: > On 03/25/2014 04:17 PM, Steven Schlansker wrote: >> >> On Mar 25, 2014, at 4:02 PM, Adrian Klaver wrote: >> >>> On 03/25/2014 03:54 PM, Steven Schlansker wrote: On Mar 25, 2014, at 3:52 PM, Adrian Klaver wrote: >

[GENERAL] A guide about some topics of Associate Certification

2014-03-25 Thread Oscar Calderon
Everybody have a nice day. Well, finally the place where i currently work paid me a chance to take the Associate Certification exam and i'm reviewing some topics, specifically the topics that the exam covers (some of them are easy to me like psql, which i use almost everyday) but i have doubt about

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Adrian Klaver
On 03/25/2014 04:17 PM, Steven Schlansker wrote: On Mar 25, 2014, at 4:02 PM, Adrian Klaver wrote: On 03/25/2014 03:54 PM, Steven Schlansker wrote: On Mar 25, 2014, at 3:52 PM, Adrian Klaver wrote: On 03/25/2014 01:56 PM, Steven Schlansker wrote: Hi everyone, I have a Postgres 9.3.3 da

Re: [GENERAL] Failure upgrading PG 9.2 to 9.3

2014-03-25 Thread Tom Lane
Sam Saffron writes: > I can confirm corrupt db upgrades fine with pg_dump. Was wondering if > there are any plans to add a --no-validate to pg_upgrade, since the > crash seems only to happen during validation. Skipping validation would probably just result in the same error happening later, when

Re: [GENERAL] Failure upgrading PG 9.2 to 9.3

2014-03-25 Thread Sam Saffron
Thanks heaps Tom, I can confirm corrupt db upgrades fine with pg_dump. Was wondering if there are any plans to add a --no-validate to pg_upgrade, since the crash seems only to happen during validation. Cheers Sam On Wed, Mar 26, 2014 at 3:19 AM, Tom Lane wrote: > Sam Saffron writes: >> Why wou

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Steven Schlansker
On Mar 25, 2014, at 4:02 PM, Adrian Klaver wrote: > On 03/25/2014 03:54 PM, Steven Schlansker wrote: >> >> On Mar 25, 2014, at 3:52 PM, Adrian Klaver wrote: >> >>> On 03/25/2014 01:56 PM, Steven Schlansker wrote: Hi everyone, I have a Postgres 9.3.3 database machine. Due to s

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Adrian Klaver
On 03/25/2014 03:54 PM, Steven Schlansker wrote: On Mar 25, 2014, at 3:52 PM, Adrian Klaver wrote: On 03/25/2014 01:56 PM, Steven Schlansker wrote: Hi everyone, I have a Postgres 9.3.3 database machine. Due to some intelligent work on the part of someone who shall remain nameless, the WAL

Re: [GENERAL] PG choosing nested loop for set membership?

2014-03-25 Thread Tom Lane
Brian Crowell writes: > Explaining just this view yields: > 'Nested Loop (cost=2.77..10.23 rows=2 width=10) (actual time=0.086..0.222 > rows=241 loops=1)' > ' -> Hash Right Join (cost=2.62..5.12 rows=1 width=8) (actual > time=0.064..0.068 rows=1 loops=1)' > ' -> Index Scan using _visible_

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Steven Schlansker
On Mar 25, 2014, at 3:52 PM, Adrian Klaver wrote: > On 03/25/2014 01:56 PM, Steven Schlansker wrote: >> Hi everyone, >> >> I have a Postgres 9.3.3 database machine. Due to some intelligent work on >> the part of someone who shall remain nameless, the WAL archive command >> included a ‘> /dev

Re: [GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Adrian Klaver
On 03/25/2014 01:56 PM, Steven Schlansker wrote: Hi everyone, I have a Postgres 9.3.3 database machine. Due to some intelligent work on the part of someone who shall remain nameless, the WAL archive command included a ‘> /dev/null 2>&1’ which masked archive failures until the disk entirely fi

[GENERAL] Disk Encryption in Production

2014-03-25 Thread Carlos Espejo
Anybody running their PostgreSQL server from a ecryptfs container? What are the common production setups out there? What are the drawbacks that people have experienced with their solution? -- Carlos Espejo

Re: [GENERAL] Upgrading from 9.2 to 9.3 causes performance degradation

2014-03-25 Thread Alfonso Afonso
Hi Alvaro I had a look to the explain analyze you sent and after step 13 you have some differences between both planners. At 21st step you have a sequence scan on 9.3 instead of a index scan on 19th step (9.2) Following the process, you can find more differences too (45 & 43, seq scan salario

Re: [GENERAL] tsearch2: how to plainto_tsquery() with "|"

2014-03-25 Thread John Smith
i thought you shared my frustration :-) (see http://postgresql.1045698.n5.nabble.com/tsearch2-plainto-tsquery-with-OR-td1885955.html). anyway, then "...plainto_tsquery('...')" is pretty much useless when it fails if someone inserts a single boolean operator - back to "...to_tsquery('...')" and inse

Re: [GENERAL] PG choosing nested loop for set membership?

2014-03-25 Thread Brian Crowell
On Tue, Mar 25, 2014 at 4:12 PM, David Johnston wrote: > 2 - and the one I'd use by default) Use an INNER JOIN That's where I started, but Postgres is smart enough to know that this is equivalent to what I'm doing, and still picks the nested loop. I went to IN in the hopes of tricking it. I have

Re: [GENERAL] PG choosing nested loop for set membership?

2014-03-25 Thread Brian Crowell
On Tue, Mar 25, 2014 at 4:07 PM, Tom Lane wrote: > So the main estimation error is inside that view, which you didn't > show us :-( I didn't know which direction you'd want to go with it. :P The view is like this: === create or replace view pl2.visible_accounts as select -- {{pk}} The acco

Re: [GENERAL] PG choosing nested loop for set membership?

2014-03-25 Thread David Johnston
Brian Crowell wrote > Hello, it's me, a Postgres n00b again. I'm dealing with a query that > scans a rather large table (94,000,000 tuples or so) and just picks > out certain rows and sums them: > > select dci.snapshot_time as "time", round(sum(dci.todays_pl)::numeric,0) > as pl > from dbo._pl_dat

Re: [GENERAL] PG choosing nested loop for set membership?

2014-03-25 Thread Tom Lane
Brian Crowell writes: > The trouble comes when enforcing the "dci.account in (...)" search > condition: pl2.visible_accounts is a view that determines which > accounts the current user can see, which, depending on who you are, > can be several hundred or none at all. Postgres estimates the output

[GENERAL] PG choosing nested loop for set membership?

2014-03-25 Thread Brian Crowell
Hello, it's me, a Postgres n00b again. I'm dealing with a query that scans a rather large table (94,000,000 tuples or so) and just picks out certain rows and sums them: select dci.snapshot_time as "time", round(sum(dci.todays_pl)::numeric,0) as pl from dbo._pl_data_cache_intraday dci where dci.sna

[GENERAL] Trimming transaction logs after extended WAL archive failures

2014-03-25 Thread Steven Schlansker
Hi everyone, I have a Postgres 9.3.3 database machine. Due to some intelligent work on the part of someone who shall remain nameless, the WAL archive command included a ‘> /dev/null 2>&1’ which masked archive failures until the disk entirely filled with 400GB of pg_xlog entries. I have fixed

Re: [GENERAL] tsearch2: how to plainto_tsquery() with "|"

2014-03-25 Thread Tom Lane
John Smith writes: > i can run "...@@ to_tsquery('cat | dog')". > but if i run "...@@ to_tsquery('cat dog')", it gives me a syntax error > (#42601). > so i run "...@@ plainto_tsquery('cat dog')". > but then i can't run "...@@ plainto_tsquery('cat | dog')". Yeah ... that's pretty much exactly the

[GENERAL] tsearch2: how to plainto_tsquery() with "|"

2014-03-25 Thread John Smith
i can run "...@@ to_tsquery('cat | dog')". but if i run "...@@ to_tsquery('cat dog')", it gives me a syntax error (#42601). so i run "...@@ plainto_tsquery('cat dog')". but then i can't run "...@@ plainto_tsquery('cat | dog')". can you help before i give up on tsearch2? thks, jzs http://postg

Re: [GENERAL] Upgrading from 9.2 to 9.3 causes performance degradation

2014-03-25 Thread Joseph Kregloh
I browsed through messages related to this issue? I have a few questions to the OP. How did you upgrade? Did you reindex the database? Try reindexdb and see what happens. If you used pg_upgrade, you need to reindex. I did pg_upgrade on my production db and it wasn't any faster on 9.3 than on 9.0, u

Re: [GENERAL] Failure upgrading PG 9.2 to 9.3

2014-03-25 Thread Tom Lane
Sam Saffron writes: > Why would > "ERROR: operator does not exist: name !~ unknown" > Come up ? It's hard to explain that as anything except corrupted system catalogs in your existing database :-(. If you were really lucky, reindexing pg_operator would fix it; but since pg_operator is usually p

Re: [GENERAL] Is it possible to "pip" pg_dump output into new db ?

2014-03-25 Thread Alan Hodgson
On Tuesday, March 25, 2014 02:56:48 PM Frank Foerster wrote: > Hi, > > we are currently in the process of upgrading a production/live 1 TB > database from 9.2 to 9.3 via pg_dump, which is quite a lengthy process. > > Fortunately we have a capable spare-server so we can restore into a clean, > fre

Re: [GENERAL] Failure upgrading PG 9.2 to 9.3

2014-03-25 Thread Adrian Klaver
On 03/24/2014 04:58 PM, Sam Saffron wrote: I am getting the following failure on a customer DB upgrading 9.2 to 9.3 Selecting previously unselected package postgresql-9.2. Unpacking postgresql-9.2 (from .../postgresql-9.2_9.2.8-1.pgdg12.4+1_amd64.deb) ... Processing triggers for postgresql-commo

Re: [GENERAL] Is it possible to "pip" pg_dump output into new db ?

2014-03-25 Thread Raymond O'Donnell
On 25/03/2014 13:56, Frank Foerster wrote: > > Hi, > > we are currently in the process of upgrading a production/live 1 TB > database from 9.2 to 9.3 via pg_dump, which is quite a lengthy process. > > Fortunately we have a capable spare-server so we can restore into a > clean, freshly setup mac

[GENERAL] Please ignore my previous mail: piping pg_dump output / solved

2014-03-25 Thread Frank
Hi, please ignore my previous question about piping pg_dump output to pg_restore. This is already working. Thanks, Frank

[GENERAL] Is it possible to "pip" pg_dump output into new db ?

2014-03-25 Thread Frank Foerster
Hi, we are currently in the process of upgrading a production/live 1 TB database from 9.2 to 9.3 via pg_dump, which is quite a lengthy process. Fortunately we have a capable spare-server so we can restore into a clean, freshly setup machine. I just wondered wether the intermediate step of writin

[GENERAL] Failure upgrading PG 9.2 to 9.3

2014-03-25 Thread Sam Saffron
I am getting the following failure on a customer DB upgrading 9.2 to 9.3 Selecting previously unselected package postgresql-9.2. Unpacking postgresql-9.2 (from .../postgresql-9.2_9.2.8-1.pgdg12.4+1_amd64.deb) ... Processing triggers for postgresql-common ... Setting up postgresql-client-9.2 (9.2.8

Re: [GENERAL] insert data over dblink from bulk select of local table

2014-03-25 Thread Adrian Klaver
On 03/25/2014 07:22 AM, Marcus Engene wrote: Hi, I have a production database with some heavy views. I'd like to make a hourly cron (or smth) that bulk inserts new entries into a table on RDS. It's easy to get data down to the prod side, but is it possible to do a bulk insert the other way aro

Re: [GENERAL] Upgrading from 9.2 to 9.3 causes performance degradation

2014-03-25 Thread Adrian Klaver
On 03/24/2014 07:32 PM, Álvaro Nunes Lemos Melo wrote: Jeff: I ran a VACUUM FULL ANALYZE and retested. No significative change was notices, the explain is availiable in 9.3 - After VACUUM FULL ANALYZE - http://explain.depesz.com/s/rVoW 9.3 - Original - http://explain.depesz.com/s/Vwt Adrian: I'

[GENERAL] insert data over dblink from bulk select of local table

2014-03-25 Thread Marcus Engene
Hi, I have a production database with some heavy views. I'd like to make a hourly cron (or smth) that bulk inserts new entries into a table on RDS. It's easy to get data down to the prod side, but is it possible to do a bulk insert the other way around? All examples I've seen adds the data

Re: [GENERAL] Strange result for full text query

2014-03-25 Thread Tom Lane
Johann Spies writes: > I have a record with this in the 'ti' field: > On type A and type B uncertainties and its propagation without > derivatives: a contribution to incorporate contemporary metrology to > Physics' laboratories in higher education > and tsv contains the following for this record

[GENERAL] Strange result for full text query

2014-03-25 Thread Johann Spies
I have a record with this in the 'ti' field: On type A and type B uncertainties and its propagation without derivatives: a contribution to incorporate contemporary metrology to Physics' laboratories in higher education And there is a tsv-field which is updated by the following trigger function: