Re: [HACKERS] Removal of useless include references

2011-08-25 Thread Christian Ullrich
* Bruce Momjian wrote: The attached patch removes unneeded include references, and marks some includes as needing to be skipped by pgrminclude. There are several unrelated changes to pg_upgrade in that patch, too. -- Christian -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] patch to slightly improve clarity of a comment in postgresql.conf.sample

2011-08-25 Thread Dougal Sutherland
Sure, I like that more. On Thu, Aug 25, 2011 at 9:29 AM, Robert Haas wrote: > On Wed, Aug 24, 2011 at 7:33 AM, Dougal Sutherland > wrote: > > The attached change to postgresql.conf.sample makes it more clear at a > > glance that the default value of listen_addresses is 'localhost', not > > 'lo

Re: [HACKERS] tsvector concatenation - backend crash

2011-08-25 Thread Jesper Krogh
On 2011-08-26 05:28, Tom Lane wrote: Jesper Krogh writes: Attached SQL files gives (at least in my hands) a reliable backend crash with this stacktrace .. reproduced on both 9.0.4 and HEAD. I'm sorry I cannot provide a more trimmed down set of vectors the reproduces the bug, thus the "obsfucate

Re: [HACKERS] Buffering GiST leaf pages too

2011-08-25 Thread Heikki Linnakangas
On 26.08.2011 00:45, Jim Nasby wrote: I've often wondered about the per-tuple overhead of all kinds of operations, not just GiST index builds. For example, if you're doing a seqscan, ISTM it would be a lot more efficient to memcpy an entire page into backend-local memory and operate off of tha

Re: [HACKERS] Removal of useless include references

2011-08-25 Thread Tom Lane
Bruce Momjian writes: > It has been years since I ran src/tools/pginclude/pgrminclude to remove > unnecessary include files. (I have already fixed things so include > files can be compiled on their own.) > The attached patch removes unneeded include references, and marks some > includes as needi

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-25 Thread Bruce Momjian
OK, this was very helpful. I found out that there is a bug in current 9.0.X, 9.1.X, and HEAD that I introduced recently when I excluded temp tables. (The bug is not in any released version of pg_upgrade.) The attached, applied patches should fix it for you. I assume you are running 9.0.X, and

[HACKERS] dropdb and dropuser: IF EXISTS

2011-08-25 Thread Josh Kupershmidt
I noticed a few places where it would be handy if dropdb took a flag like "--if-exists" which would basically just add in the 'IF EXISTS' clause to the DROP DATABASE statement. For example, scripts like find_static or mbregress.sh use dropdb && createdb, but they generate noisy errors from dropdb w

Re: [HACKERS] tsvector concatenation - backend crash

2011-08-25 Thread Tom Lane
Jesper Krogh writes: > Attached SQL files gives (at least in my hands) a reliable backend crash > with this stacktrace .. reproduced on both 9.0.4 and HEAD. I'm sorry > I cannot provide a more trimmed down set of vectors the reproduces the > bug, thus > the "obsfucated" dataset. But even deleting

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 25, 2011 at 5:12 PM, Tom Lane wrote: >> ISTM the general rule ought to be that we attempt to substitute for a >> colon-construct regardless of where it appears within an argument, as >> long as it's not within quotes. > My main thought is that I remember this co

Re: [HACKERS] Why doesn't psql use the information schema to get ACL description ?

2011-08-25 Thread Dave Cramer
On Thu, Aug 25, 2011 at 5:51 PM, Jim Nasby wrote: > On Aug 23, 2011, at 1:57 PM, Peter Eisentraut wrote: >>> Ultimately my ulterior motive is to make sure the information schema >>> is actually useful and ensure that it maintains that status. >> >> The information schema only exposes privileges de

[HACKERS] pg_upgrade problem (fwd)

2011-08-25 Thread Bruce Momjian
I am sending this report to hackers to all hacker subscribers can read the original bug report. --- - Forwarded message from hubert depesz lubaczewski - hi I have 8.3.11 database, ~ 600GB in size. I want to upgrad

Re: [HACKERS] PATCH: regular logging of checkpoint progress

2011-08-25 Thread Tomas Vondra
On 26 Srpen 2011, 0:39, Tomas Vondra wrote: > On 26 Srpen 2011, 0:18, Josh Berkus wrote: >> Tomas, >> >>> I'd like to propose a small patch that allows better checkpoint >>> progress >>> monitoring. The patch is quite simple - it adds a new integer GUC >>> "checkpoint_update_limit" and every time c

Re: [HACKERS] PATCH: regular logging of checkpoint progress

2011-08-25 Thread Tomas Vondra
On 26 Srpen 2011, 0:18, Josh Berkus wrote: > Tomas, > >> I'd like to propose a small patch that allows better checkpoint progress >> monitoring. The patch is quite simple - it adds a new integer GUC >> "checkpoint_update_limit" and every time checkpoint writes this number >> of >> buffers, it does

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Jim Nasby
On Aug 22, 2011, at 6:22 PM, Robert Haas wrote: > With respect to a general-purpose shared memory allocator, I think > that there are cases where that would be useful to have, but I don't > think there are as many of them as many people seem to think. I > wouldn't choose to implement this using a

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Jim Nasby
On Aug 25, 2011, at 8:24 AM, Robert Haas wrote: > My hope (and it might turn out that I'm an optimist) is that even with > a reasonably small buffer it will be very rare for a backend to > experience a wraparound condition. For example, consider a buffer > with ~6500 entries, approximately 64 * Ma

Re: [HACKERS] Inputting relative datetimes

2011-08-25 Thread Josh Berkus
> Yes, but what I am trying to achieve is a way of entering such > relative timestamps using a single input value, so that absolute and > relative timestamps can both be bound to a SQL query using just one > variable. I think adding a function would be the way to go then. Maybe extending to_time

Re: [HACKERS] PATCH: regular logging of checkpoint progress

2011-08-25 Thread Josh Berkus
Tomas, > I'd like to propose a small patch that allows better checkpoint progress > monitoring. The patch is quite simple - it adds a new integer GUC > "checkpoint_update_limit" and every time checkpoint writes this number of > buffers, it does two things: I'd rather not have a new GUC if we can

Re: [HACKERS] pg_restore --no-post-data and --post-data-only

2011-08-25 Thread Andrew Dunstan
On 08/25/2011 06:05 PM, Jim Nasby wrote: On Aug 24, 2011, at 7:43 PM, Josh Berkus wrote: On 8/23/11 1:30 PM, Andrew Dunstan wrote: Attached is an undocumented patch that allows pg_restore to omit post-data items or omit all but post-data items. This has been discussed before, and Simon sent i

Re: [HACKERS] pg_restore --no-post-data and --post-data-only

2011-08-25 Thread Jim Nasby
On Aug 24, 2011, at 7:43 PM, Josh Berkus wrote: > On 8/23/11 1:30 PM, Andrew Dunstan wrote: >> >> Attached is an undocumented patch that allows pg_restore to omit >> post-data items or omit all but post-data items. This has been discussed >> before, and Simon sent in a patch back on 2008, which ha

Re: [HACKERS] Why doesn't psql use the information schema to get ACL description ?

2011-08-25 Thread Jim Nasby
On Aug 23, 2011, at 1:57 PM, Peter Eisentraut wrote: >> Ultimately my ulterior motive is to make sure the information schema >> is actually useful and ensure that it maintains that status. > > The information schema only exposes privileges defined in the SQL > standard on objects defined in the SQ

Re: [HACKERS] Buffering GiST leaf pages too

2011-08-25 Thread Jim Nasby
On Aug 23, 2011, at 2:03 AM, Heikki Linnakangas wrote: > While looking at Alexander's GiST fastbuild patch, which adds buffers to > internal nodes to avoid random I/O during index build, it occurred to me that > inserting the tuples to the leaf pages one at a time is quite inefficient > too, eve

Re: [HACKERS] Inputting relative datetimes

2011-08-25 Thread Merlin Moncure
On Thu, Aug 25, 2011 at 5:08 AM, Dean Rasheed wrote: > On 25 August 2011 10:43, Vik Reykja wrote: >> On Thu, Aug 25, 2011 at 11:39, Dean Rasheed >> wrote: >>> >>> My first thought was to have some general way of adding or subtracting >>> an interval at the end of an input timestamp, eg. by addin

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Andrew Dunstan
On 08/25/2011 02:45 PM, Tom Lane wrote: I wrote: Andrew Dunstan writes: We could just add -b unconditionally to the flex flags and then count the number of lines in lex.backup. If it's greater that 1 whine loudly, or even fail, otherwise remove lex.backup. Would that avoid locale dependencie

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 5:12 PM, Tom Lane wrote: > While I'm looking at this ... the current implementation has got a > number of very inconsistent behaviors with respect to when it will > expand a variable reference within a psql meta-command argument. > Observe: > > regression=# \set foo 'value

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
While I'm looking at this ... the current implementation has got a number of very inconsistent behaviors with respect to when it will expand a variable reference within a psql meta-command argument. Observe: regression=# \set foo 'value of foo' regression=# \set bar 'value of bar' regression=# \ec

[HACKERS] PATCH: regular logging of checkpoint progress

2011-08-25 Thread Tomas Vondra
Hello, I'd like to propose a small patch that allows better checkpoint progress monitoring. The patch is quite simple - it adds a new integer GUC "checkpoint_update_limit" and every time checkpoint writes this number of buffers, it does two things: (a) logs a "checkpoint status" message into the

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-25 Thread hubert depesz lubaczewski
On Thu, Aug 25, 2011 at 04:43:02PM -0400, Bruce Momjian wrote: > Please check the old cluster. Sure: =# SELECT reltoastrelid FROM pg_class WHERE relname = 'actions';

Re: [HACKERS] SSI 2PC coverage

2011-08-25 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mié ago 24 22:11:58 -0300 2011: > Alvaro Herrera writes: > > After having to play with this, I didn't like it very much, because > > regression.diffs gets spammed with the (rather massive and completely > > useless) diff in that test. For the xml tests, rather

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-25 Thread Bruce Momjian
hubert depesz lubaczewski wrote: > On Thu, Aug 25, 2011 at 04:33:07PM -0400, Bruce Momjian wrote: > > The problem appears to be that the Postgres catalogs think there is a > > toast table for 'actions', while the file system doesn't seem to have > > such a file. I can you look in pg_class and veri

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-25 Thread hubert depesz lubaczewski
On Thu, Aug 25, 2011 at 04:33:07PM -0400, Bruce Momjian wrote: > The problem appears to be that the Postgres catalogs think there is a > toast table for 'actions', while the file system doesn't seem to have > such a file. I can you look in pg_class and verify that? > > SELECT reltoastrelid

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-25 Thread Bruce Momjian
hubert depesz lubaczewski wrote: > hi > > I have 8.3.11 database, ~ 600GB in size. > > I want to upgrade it to 9.0. > > First, I tried with 9.0.4, and when I hit problem (the same) I tried > git, head of 9.0 branch. Good. > pg_upgrade_dump_db.sql- > pg_upgrade_dump_db.sql--- For binary upgrade

Re: [HACKERS] [v9.1] sepgsql - userspace access vector cache

2011-08-25 Thread Robert Haas
On Fri, Aug 5, 2011 at 2:36 PM, Kohei KaiGai wrote: > BTW, what is the current status of this patch? > The status of contrib/sepgsql part is unclear for me, although we agreed that > syscache is suitable mechanism for security labels. Sorry it's taken me a while to get around to looking at this.

[HACKERS] tsvector concatenation - backend crash

2011-08-25 Thread Jesper Krogh
Hi Attached SQL files gives (at least in my hands) a reliable backend crash with this stacktrace .. reproduced on both 9.0.4 and HEAD. I'm sorry I cannot provide a more trimmed down set of vectors the reproduces the bug, thus the "obsfucated" dataset. But even deleting single terms in the vecto

Re: [HACKERS] Backup's from standby

2011-08-25 Thread Robert Haas
On Fri, Aug 19, 2011 at 9:38 AM, Valentine Gogichashvili wrote: >> > What issue we may face if you take a backups(includes data dir + wal >> > files) >> > at standby without LVM snapshot? >> >> The backup might be corrupted in arbitrary ways. > > And what will happen, if one issues a pg_start_back

Re: [HACKERS] WIP: Fast GiST index build

2011-08-25 Thread Heikki Linnakangas
On 24.08.2011 16:57, Alexander Korotkov wrote: I've added some testing results to the wiki page: http://wiki.postgresql.org/wiki/Fast_GiST_index_build_GSoC_2011 There are not all the results I planned for the first chunk because it takes more time than I expect. Some notes about it. Now I see tw

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
I wrote: > Andrew Dunstan writes: >> We could just add -b unconditionally to the flex flags and then count >> the number of lines in lex.backup. If it's greater that 1 whine loudly, >> or even fail, otherwise remove lex.backup. Would that avoid locale >> dependencies? > Hm, yeah, seems like th

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
Andrew Dunstan writes: > We could just add -b unconditionally to the flex flags and then count > the number of lines in lex.backup. If it's greater that 1 whine loudly, > or even fail, otherwise remove lex.backup. Would that avoid locale > dependencies? Hm, yeah, seems like that ought to work.

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Andrew Dunstan
On 08/25/2011 01:16 PM, Alvaro Herrera wrote: Excerpts from Tom Lane's message of jue ago 25 14:00:57 -0300 2011: Robert Haas writes: On Thu, Aug 25, 2011 at 12:47 PM, Tom Lane wrote: 1. Somebody broke the no-backtracking property back in 9.0 while adding quoted variable substitution. Acc

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Alvaro Herrera
Excerpts from Tom Lane's message of jue ago 25 14:00:57 -0300 2011: > Robert Haas writes: > > On Thu, Aug 25, 2011 at 12:47 PM, Tom Lane wrote: > >> 1. Somebody broke the no-backtracking property back in 9.0 while adding > >> quoted variable substitution. According to the flex manual, use of > >

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 25, 2011 at 12:47 PM, Tom Lane wrote: >> 1. Somebody broke the no-backtracking property back in 9.0 while adding >> quoted variable substitution. According to the flex manual, use of >> backtracking creates a performance penalty. We once measured the >> backend

Re: [HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 12:47 PM, Tom Lane wrote: > On my way to do something else entirely, I came across a couple of > things that are not very nice about psql's lexer: > > 1. Somebody broke the no-backtracking property back in 9.0 while adding > quoted variable substitution.  According to the f

Re: [HACKERS] Change format of FDW options used in \d* commands

2011-08-25 Thread Robert Haas
2011/8/19 Shigeru Hanada : > Oops, good catch.  I've fixed psql to use quote_ident() for option_name, > and modified regression tests to use special characters in option names. >  Please try revised version of patch. This part looks good to me, too. Committed. -- Robert Haas EnterpriseDB: http:

[HACKERS] A couple of issues with psql variable substitution

2011-08-25 Thread Tom Lane
On my way to do something else entirely, I came across a couple of things that are not very nice about psql's lexer: 1. Somebody broke the no-backtracking property back in 9.0 while adding quoted variable substitution. According to the flex manual, use of backtracking creates a performance penalt

Re: [HACKERS] Change format of FDW options used in \d* commands

2011-08-25 Thread Robert Haas
2011/8/19 Shigeru Hanada : > BTW, I noticed that pg_dump has same issue since 8.4, initial release of > SQL/MED infrastructure.  If a FDW option was defined on any FDW object > with a name which contains one of special characters such as space and > parentheses, pg_dump generates invalid OPTIONS cl

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 11:15 AM, Markus Wanner wrote: > On 08/25/2011 04:59 PM, Tom Lane wrote: >> That's a good point.  If the ring buffer size creates a constraint on >> the maximum number of sub-XIDs per transaction, you're going to need a >> fallback path of some sort. > > I think Robert envi

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
Tom, On 08/25/2011 04:59 PM, Tom Lane wrote: > That's a good point. If the ring buffer size creates a constraint on > the maximum number of sub-XIDs per transaction, you're going to need a > fallback path of some sort. I think Robert envisions the same fallback path we already have: subxids.over

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
Robert, On 08/25/2011 04:48 PM, Robert Haas wrote: > What's a typical message size for imessages? Most message types in Postgres-R are just a couple bytes in size. Others, especially change sets, can be up to 8k. However, I think you'll have an easier job guaranteeing that backends "consume" the

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Tom Lane
Robert Haas writes: > Well, one long-running transaction that only has a single XID is not > really a problem: the snapshot is still small. But one very old > transaction that also happens to have a large number of > subtransactions all of which have XIDs assigned might be a good way to > stress

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 10:19 AM, Markus Wanner wrote: > Note, however, that for imessages, I've also had the policy in place > that a backend *must* consume its message before sending any.  And that > I took great care for all receivers to consume their messages as early > as possible.  None the

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Markus Wanner
Robert, On 08/25/2011 03:24 PM, Robert Haas wrote: > My hope (and it might turn out that I'm an optimist) is that even with > a reasonably small buffer it will be very rare for a backend to > experience a wraparound condition. It certainly seems less likely than with the ring-buffer for imessages

Re: [HACKERS] patch to slightly improve clarity of a comment in postgresql.conf.sample

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 9:30 AM, Dougal Sutherland wrote: > Sure, I like that more. OK, done. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Short document fix

2011-08-25 Thread Robert Haas
On Tue, Aug 23, 2011 at 4:53 PM, Hitoshi Harada wrote: > In the CREATE DOMAIN reference page of the current HEAD, it says > > --- > CREATE DOMAIN us_postal_code AS TEXT > CHECK( >   VALUE ~ '^\\d{5}$' > OR VALUE ~ '^\\d{5}-\\d{4}$' > ); > --- > > but I believe it should conform the standard string

Re: [HACKERS] patch to slightly improve clarity of a comment in postgresql.conf.sample

2011-08-25 Thread Robert Haas
On Wed, Aug 24, 2011 at 7:33 AM, Dougal Sutherland wrote: > The attached change to postgresql.conf.sample makes it more clear at a > glance that the default value of listen_addresses is 'localhost', not > 'localhost, *'. This would have saved a friend an hour or two of fiddling > tonight. How abo

Re: [HACKERS] cheaper snapshots redux

2011-08-25 Thread Robert Haas
On Thu, Aug 25, 2011 at 1:55 AM, Markus Wanner wrote: >> One difference with snapshots is that only the latest snapshot is of >> any interest. > > Theoretically, yes.  But as far as I understood, you proposed the > backends copy that snapshot to local memory.  And copying takes some > amount of ti

Re: [HACKERS] Inputting relative datetimes

2011-08-25 Thread Dean Rasheed
On 25 August 2011 10:43, Vik Reykja wrote: > On Thu, Aug 25, 2011 at 11:39, Dean Rasheed > wrote: >> >> My first thought was to have some general way of adding or subtracting >> an interval at the end of an input timestamp, eg. by adding another >> couple of special values - "plus " and "minus ".

Re: [HACKERS] Inputting relative datetimes

2011-08-25 Thread Vik Reykja
On Thu, Aug 25, 2011 at 11:39, Dean Rasheed wrote: > My first thought was to have some general way of adding or subtracting > an interval at the end of an input timestamp, eg. by adding another > couple of special values - "plus " and "minus ". > This would allow things like: > > TIMESTAMPTZ 'toda

[HACKERS] Inputting relative datetimes

2011-08-25 Thread Dean Rasheed
As background, I have an app that accepts user text input and casts it to a timestamp in order to produce reports. I use PostgreSQL's timestamp input conversion for this, since it gives a lot of flexibility, and can parse pretty much anything the users throw at it. It is also handy that it recogni