Re: [GENERAL] Comparing txid_current() to xmin

2012-11-07 Thread Andres Freund
On Tue, Nov 06, 2012 at 02:55:40PM -0800, Mike Lewis wrote: I am trying to make a trigger that updates a row once and only once per transaction (even if this trigger gets fired multiple times). The general idea is that for a user we have a version number. When we modify the user's data, the

Re: [GENERAL] Memory issue on FreeBSD

2012-11-07 Thread Achilleas Mantzios
Vick, fantastic script, thanx! FreeBSD sysctl system is awesome! On Τρι 06 Νοε 2012 14:33:43 Vick Khera wrote: On Mon, Nov 5, 2012 at 10:11 AM, Frank Broniewski b...@metrico.lu wrote: and this is after a few hours of running: Mem: 91M Active, 17G Inact, 3983M Wired, 1526M Cache, 3283M Buf,

Re: [GENERAL] Memory issue on FreeBSD

2012-11-07 Thread Frank Broniewski
Hey, this is really cool. I directly tried the script and there's a line from the output that caught my eye: mem_gap_vm: + 8812892160 ( 8404MB) [ 26%] Memory gap: UNKNOWN is this the shared buffers? I guess so, but I want to confirm my guess ... Frank Am 2012-11-07 09:26, schrieb

[GENERAL] How to evaluate explain analyze correctly soon after explain for the same statement ?

2012-11-07 Thread 高健
Hi all: I have one question about the cache clearing. If I use the following soon after database startup(or first time I use it): postgres=# explain analyze select id,deptno from gaotab where id=200; QUERY PLAN

Re: [GENERAL] Memory issue on FreeBSD

2012-11-07 Thread Achilleas Mantzios
On Τετ 07 Νοε 2012 09:42:47 Frank Broniewski wrote: Hey, this is really cool. I directly tried the script and there's a line from the output that caught my eye: mem_gap_vm: + 8812892160 ( 8404MB) [ 26%] Memory gap: UNKNOWN is this the shared buffers? I guess so, but I want to

Re: [GENERAL] How to evaluate explain analyze correctly soon after explain for the same statement ?

2012-11-07 Thread Albe Laurenz
高健 wrote: I have one question about the cache clearing. If I use the following soon after database startup(or first time I use it): postgres=# explain analyze select id,deptno from gaotab where id=200; The result is: the above explain analyze got a total runtime of 47 ms. But If I

Re: [GENERAL] How to evaluate explain analyze correctly soon after explain for the same statement ?

2012-11-07 Thread Tom Lane
=?UTF-8?B?6auY5YGl?= luckyjack...@gmail.com writes: It might not be a big problem in a small system. But when in a production environment, When I want to use explain and then , soon use explain analyze for the same statement, How can I avoid the influence of cache and get the right answer

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Jeff Janes
On Tue, Nov 6, 2012 at 11:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: On Tue, Nov 6, 2012 at 10:49 AM, Lists li...@benjamindsmith.com wrote: I followed your example, the result is at the bottom. Based on this it would seem that there are 3-4 databases that

Re: [GENERAL] Postgres no longer starts

2012-11-07 Thread markalcock
came across this problem myself. turned out after much playing around that it was a change to the pg_hba.conf was a syntax error in the all all posgres trust sameuser line. deleted it and postgres fired up from /etc/init.d or as a service. just my very late twopenneth -- View this

Re: [GENERAL] Memory issue on FreeBSD

2012-11-07 Thread Achilleas Mantzios
Vick, fantastic script, thanx! FreeBSD sysctl system is awesome! On Τρι 06 Νοε 2012 14:33:43 Vick Khera wrote: On Mon, Nov 5, 2012 at 10:11 AM, Frank Broniewski b...@metrico.lu wrote: and this is after a few hours of running: Mem: 91M Active, 17G Inact, 3983M Wired, 1526M Cache, 3283M Buf,

Re: [GENERAL] Parallel Insert and Delete operation

2012-11-07 Thread Yelai, Ramkumar IN BLR STS
Ramkumar Yelai wrote: [is worried that a database might become inconsistent if conflicting INSERTs and DELETEs occur] @Albe - I got you first point. The second point is little skeptical because postgres could have been avoided this lock by using MVCC. Please correct me if I am wrong? Which

Re: [GENERAL] How to verify pg_dump files

2012-11-07 Thread Raymond O'Donnell
On 07/11/2012 13:01, Gary wrote: Can anyone suggest how I could verify that the files created by pg_dump are okay? They are being created for backup purposes, and the last thing I want to do is find out that the backups themselves are in some way corrupt. I know I can check the output of

Re: [GENERAL] Plug-pull testing worked, diskchecker.pl failed

2012-11-07 Thread Bruce Momjian
On Sat, Oct 27, 2012 at 05:41:02PM +1100, Chris Angelico wrote: On Sat, Oct 27, 2012 at 4:26 PM, Greg Smith g...@2ndquadrant.com wrote: In general, through, diskchecker.pl is the more sensitive test. If it fails, storage is unreliable for PostgreSQL, period. It's good that you've

Re: [GENERAL] 9.1 to 9.2 requires a dump/reload?

2012-11-07 Thread Bruce Momjian
On Tue, Oct 23, 2012 at 09:41:20AM -0400, Nikolas Everett wrote: On Mon, Oct 22, 2012 at 6:17 PM, Alan Hodgson ahodg...@simkin.ca wrote: On Monday, October 22, 2012 05:55:07 PM Nikolas Everett wrote: I see that pg_upgrade is an option. Having never used how long should I

Re: [GENERAL] help with upgrade from 9.1 to 9.2

2012-11-07 Thread Bruce Momjian
On Thu, Oct 25, 2012 at 02:39:09AM -0200, Aníbal Pacheco wrote: I could after some work, what I want to ask now is this: In the middle of the pg_restore process I had to stop it (^Z) and remove one problematic and not needed database from the generated pg_upgrade_dump_db.sql file and then

Re: [GENERAL] Extra space when converting number with to_char

2012-11-07 Thread Bruce Momjian
On Fri, Oct 26, 2012 at 04:11:42PM -0400, Samuel Gilbert wrote: Thank you, it works. The documentation gave me the impression that the FM modifier only applied to date/time since it was under Usage notes for date/time formatting: Uh, I see: entryliteralFM/literal prefix/entry

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Lists
On 11/07/2012 09:01 AM, Jeff Janes wrote: Ben, did you ever figure out where the space was going? I think we've found where the space is going, but I still don't yet know how to resolve it. I modified your query thusly in order to get a total of space used, and got an answer that matches

Re: [GENERAL] How to verify pg_dump files

2012-11-07 Thread Igor Neyman
-Original Message- From: Gary [mailto:listgj...@yahoo.co.uk] Sent: Wednesday, November 07, 2012 8:02 AM To: pgsql-general@postgresql.org Subject: How to verify pg_dump files Can anyone suggest how I could verify that the files created by pg_dump are okay? They are being created

Re: [GENERAL] Why PGDLLIMPORT is needed

2012-11-07 Thread Bruce Momjian
On Mon, Oct 29, 2012 at 04:41:05PM +0800, Craig Ringer wrote: On 10/29/2012 02:05 PM, 高健 wrote: On /src/include/storage/proc.h: I saw the following line: extern PGDLLIMPORT PGPROC *MyProc; I want to know why PGDLLIMPORT is used here? Does it mean: exten PGPROC *MyProc;

Re: [GENERAL] Problem with streaming replication over SSL

2012-11-07 Thread Albe Laurenz
Magnus Hagander wrote: I have streaming replication configured over SSL, and there seems to be a problem with SSL renegotiation. [...] After that, streaming replication reconnects and resumes working. Is this an oversight in the replication protocol, or is this working as designed? This

Re: [GENERAL] How to verify pg_dump files

2012-11-07 Thread Raymond O'Donnell
On 07/11/2012 18:57, Ryan Delaney wrote: On Wed, Nov 7, 2012 at 10:11 AM, Raymond O'Donnell r...@iol.ie mailto:r...@iol.ie wrote: On 07/11/2012 13:01, Gary wrote: Can anyone suggest how I could verify that the files created by pg_dump are okay? They are being created for

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Tom Lane
Lists li...@benjamindsmith.com writes: pg_catalog.pg_attribute | 36727480320 Ouch. Our current process is that every night in the middle of the night, a script connects to each database on each server and runs a query to get all tables in each database

Re: [GENERAL] Does PostgreSQL have complete functional test cases?

2012-11-07 Thread Bruce Momjian
On Tue, Nov 6, 2012 at 09:24:19AM +0100, Pavel Stehule wrote: 2012/11/6 Tianyin Xu t...@cs.ucsd.edu: Thanks, Pavel! I see. So the regress test cases are the complete functional testing? Am I right? yes Those tests are hardly complete, as in testing every possible input and output.

Re: [GENERAL] How to verify pg_dump files

2012-11-07 Thread Ryan Delaney
On Wed, Nov 7, 2012 at 10:11 AM, Raymond O'Donnell r...@iol.ie wrote: On 07/11/2012 13:01, Gary wrote: Can anyone suggest how I could verify that the files created by pg_dump are okay? They are being created for backup purposes, and the last thing I want to do is find out that the backups

[GENERAL] Unique/Primary key not inherited in partition, workaround?

2012-11-07 Thread Nicholas Wilson
Regarding the caveats here http://www.postgresql.org/docs/9.1/interactive/ddl-inherit.html#DDL-INHERIT-CAVEATS I am attempting to logically structure my location data. Say for example I have cities/states/countries. I have objects that reference a location, but at any level. An object may

Re: [GENERAL] Plug-pull testing worked, diskchecker.pl failed

2012-11-07 Thread Scott Marlowe
On Wed, Nov 7, 2012 at 11:59 AM, Bruce Momjian br...@momjian.us wrote: On Sat, Oct 27, 2012 at 05:41:02PM +1100, Chris Angelico wrote: On Sat, Oct 27, 2012 at 4:26 PM, Greg Smith g...@2ndquadrant.com wrote: In general, through, diskchecker.pl is the more sensitive test. If it fails, storage

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Jeff Janes
Hi Greg, I've added you to the cc list because I'm proposing to change some wiki content which you wrote On Wed, Nov 7, 2012 at 11:54 AM, Lists li...@benjamindsmith.com wrote: On 11/07/2012 09:01 AM, Jeff Janes wrote: Ben, did you ever figure out where the space was going? Now, here's

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Scott Marlowe
On Wed, Nov 7, 2012 at 1:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Lists li...@benjamindsmith.com writes: ... because it occasionally causes transactions and queries to hang when an update causes a vacuum mid-day, effectively taking us offline randomly. I suspect this claim is based on

Re: [GENERAL] Plug-pull testing worked, diskchecker.pl failed

2012-11-07 Thread Bruce Momjian
On Wed, Nov 7, 2012 at 01:53:47PM -0700, Scott Marlowe wrote: On Wed, Nov 7, 2012 at 11:59 AM, Bruce Momjian br...@momjian.us wrote: On Sat, Oct 27, 2012 at 05:41:02PM +1100, Chris Angelico wrote: On Sat, Oct 27, 2012 at 4:26 PM, Greg Smith g...@2ndquadrant.com wrote: In general, through,

Re: [GENERAL] Plug-pull testing worked, diskchecker.pl failed

2012-11-07 Thread Scott Marlowe
On Wed, Nov 7, 2012 at 2:01 PM, Bruce Momjian br...@momjian.us wrote: On Wed, Nov 7, 2012 at 01:53:47PM -0700, Scott Marlowe wrote: On Wed, Nov 7, 2012 at 11:59 AM, Bruce Momjian br...@momjian.us wrote: On Sat, Oct 27, 2012 at 05:41:02PM +1100, Chris Angelico wrote: On Sat, Oct 27, 2012 at

Re: [GENERAL] Plug-pull testing worked, diskchecker.pl failed

2012-11-07 Thread Bruce Momjian
On Wed, Nov 7, 2012 at 02:12:39PM -0700, Scott Marlowe wrote: I don't have the specs to hand, but one of them is a Kingston drive. Our local supplier is out of 320 series drives, so we were looking for others; will check out the 710s. It's crazy that so few drives can actually be

Re: [GENERAL] Unique/Primary key not inherited in partition, workaround?

2012-11-07 Thread François Beausoleil
Le 2012-11-07 à 13:58, Nicholas Wilson a écrit : Regarding the caveats here http://www.postgresql.org/docs/9.1/interactive/ddl-inherit.html#DDL-INHERIT-CAVEATS I am attempting to logically structure my location data. Say for example I have cities/states/countries. I have objects that

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Lists
On 11/07/2012 12:42 PM, Tom Lane wrote: So you've turned off autovacuum, and are carefully not vacuuming the system catalogs. That's your problem all right. Is there a particularly good reason why this script isn't a one-liner VACUUM? Back in the 8.x days, we experienced vacuum full analyze

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Lists
On 11/07/2012 12:58 PM, Scott Marlowe wrote: My experience is that if autovac is causing problems with stalled queries etc you're either A: running ancient pg versions (pre 8.3), B: Running WAY too aggressive settings in autovac (100 threads, no nap time, cost limit of 10 etc.) or C: Your

Re: [GENERAL] Plug-pull testing worked, diskchecker.pl failed

2012-11-07 Thread Vick Khera
On Wed, Nov 7, 2012 at 3:53 PM, Scott Marlowe scott.marl...@gmail.comwrote: Is there a comprehensive list of drives that have been tested on the wiki somewhere? Our current choices seem to be the Intel 3xx series which STILL suffer from the whoops I'm now an 8MB drive bug and the very

Re: [GENERAL] How to verify pg_dump files

2012-11-07 Thread Lists
On 11/07/2012 11:56 AM, Igor Neyman wrote: The only 100% fool-proof test would be to restore from your backup files. Regards, Igor Neyman Our internal process is to back up production databases regularly, and then use the backups offsite to populate copies of databases for developer use.

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Greg Smith
On 11/7/12 3:58 PM, Jeff Janes wrote: WHERE nspname NOT IN ('pg_catalog', 'information_schema') I question the wisdom of that where clause (from the wiki) If the pg_catalog relations are big, then they are big and why shouldn't they get reported as such? And if they are not

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: WHERE nspname NOT IN ('pg_catalog', 'information_schema') AND C.relkind 'i' AND nspname !~ '^pg_toast' I question the wisdom of that where clause (from the wiki) If the pg_catalog relations are big, then they are big and why shouldn't they get

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Scott Marlowe
On Wed, Nov 7, 2012 at 1:58 PM, Scott Marlowe scott.marl...@gmail.com wrote: On Wed, Nov 7, 2012 at 1:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Lists li...@benjamindsmith.com writes: ... because it occasionally causes transactions and queries to hang when an update causes a vacuum mid-day,

Re: [GENERAL] Plug-pull testing worked, diskchecker.pl failed

2012-11-07 Thread David Boreham
On 11/7/2012 3:17 PM, Vick Khera wrote: My most recent big box(es) are built using all Intel 3xx series drives. Like you said, the 7xx series was way too expensive. I have to raise my hand to say that for us 710 series drives are an unbelievable bargain and we buy nothing else now for

Re: [GENERAL] Unexpectedly high disk space usage

2012-11-07 Thread Scott Marlowe
On Wed, Nov 7, 2012 at 3:15 PM, Lists li...@benjamindsmith.com wrote: On 11/07/2012 12:42 PM, Tom Lane wrote: So you've turned off autovacuum, and are carefully not vacuuming the system catalogs. That's your problem all right. Is there a particularly good reason why this script isn't a

Re: [GENERAL] How to evaluate explain analyze correctly soon after explain for the same statement ?

2012-11-07 Thread 高健
Hi tom At frist I have thought that the database parsed my explain statement, so the pre-compiled execution plan will be re-used , which made the statement's second run quick. I think that what you said is right. Thank you 2012/11/7 Tom Lane t...@sss.pgh.pa.us =?UTF-8?B?6auY5YGl?=

Re: [GENERAL] Does PostgreSQL have complete functional test cases?

2012-11-07 Thread Pavel Stehule
2012/11/7 Bruce Momjian br...@momjian.us: On Tue, Nov 6, 2012 at 09:24:19AM +0100, Pavel Stehule wrote: 2012/11/6 Tianyin Xu t...@cs.ucsd.edu: Thanks, Pavel! I see. So the regress test cases are the complete functional testing? Am I right? yes Those tests are hardly complete, as in

[GENERAL] How is execution plan cost calculated for index scan

2012-11-07 Thread 高健
Hi all: I want to see the explain plan for a simple query. My question is : How is the cost calculated? The cost parameter is: random_page_cost= 4 seq_page_cost = 1 cpu_tuple_cost =0.01 cpu_operator_cost =0.0025 And the table and its index

[GENERAL] Use order by clause, got index scan involved

2012-11-07 Thread 高健
Hi all: What confused me is that: When I select data using order by clause, I got the following execution plan: postgres=# set session enable_indexscan=true; SET postgres=# explain SELECT * FROM pg_proc ORDER BY oid; QUERY PLAN

[GENERAL] find a substring on a text (data type) column

2012-11-07 Thread pantelis vlachos
I was trying to find a substring on a text (data type) column like 'cat foo dog ...'. I use the query below SELECT id FROM table WHERE name LIKE '% foo %'; Sometimes the query return with nTuples=0 but there are matching rows. On retry, the query return with expected results. Any ideas; (postgres

Re: [GENERAL] find a substring on a text (data type) column

2012-11-07 Thread Pavel Stehule
Hello 2012/11/8 pantelis vlachos vlacho...@gmail.com: I was trying to find a substring on a text (data type) column like 'cat foo dog ...'. I use the query below SELECT id FROM table WHERE name LIKE '% foo %'; Sometimes the query return with nTuples=0 but there are matching rows. On retry,