Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-08 Thread Heikki Linnakangas

On 08.03.2011 04:07, Greg Stark wrote:

Well from that log you definitely have OldestXmin going backwards. And
not by a little bit either. at 6:33 it set the all_visible flag and
then at 7:01 it was almost 1.3 million transactions earlier. In fact
to precisely the same value that was in use for a transaction at 1:38.
That seems like a bit of a coincidence though it's not repeated
earlier.


Yep. After staring at GetOldestXmin() again, it finally struck me how 
OldestXmin can move backwards. You need two databases for it, which 
probably explains why this has been so elusive.


Here's how to reproduce that:

CREATE DATABASE foodb;
CREATE DATABASE bardb;

session 1, in foodb:

foodb=# begin isolation level serializable;
BEGIN
foodb=# CREATE TABLE foo (a int4); -- just something to force this xact 
to have an xid

CREATE TABLE
foodb=#

(leave the transaction open)

session 2, in bardb:

bardb=# CREATE TABLE foo AS SELECT 1;
SELECT
bardb=# vacuum foo; -- to set the PD_ALL_VISIBLE flag
VACUUM

session 3, in bardb:
bardb=# begin isolation level serializable;
BEGIN
bardb=# SELECT 1;
 ?column?
--
1
(1 row)

(leave transaction open)

session 2, in bardb:

bardb=# vacuum foo;
WARNING:  PD_ALL_VISIBLE flag was incorrectly set in relation foo page 
0 (OldestXmin 803)

VACUUM
bardb=#


What there are no other transactions active in the same database, 
GetOldestXmin() returns just latestCompletedXid. When you open a 
transaction in the same database after that, its xid will be above 
latestCompletedXid, but its xmin includes transactions from all 
databases, and there might be a transaction in some other database with 
an xid that precedes the value that GetOldestXmin() returned earlier.


I'm not sure what to do about that. One idea is track two xmin values in 
proc-array, one that includes transactions in all databases, and another 
that only includes transactions in the same database. GetOldestXmin() 
(when allDbs is false) would only pay attention to the latter. It would 
add a few instructions to GetSnapshotData(), though.


Another idea is to give up on the warning when it appears that 
oldestxmin has moved backwards, and assume that it's actually fine. We 
could still warn in other cases where the flag appears to be incorrectly 
set, like if there is a deleted tuple on the page.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Theory of operation of collation patch

2011-03-08 Thread Martijn van Oosterhout
On Mon, Mar 07, 2011 at 07:15:28PM -0500, Tom Lane wrote:
 Only if the expression-to-be-sorted does not already fully specify the
 collation, which so far as I can tell (either from the code or your
 description above) it does.  I think that the explicit representation
 of collation as part of the PathKey node is unnecessary, inefficient,
 and bug-inducing --- the latter because it promotes fuzzy thinking about
 where the collation information is coming from.  (And this isn't just
 hypothetical: IMO the bugs I exhibited upthread are *directly* due to
 fuzzy thinking about what defines an index's sort order.)

Well, collation processing happens in two phases. Initially collate
information is provided by the columns in the query, explicit clauses,
etc. These are indeed attached to the values. From here the collations
of expressions are determined. The SQL committee thought up a bunch of
actually quite logical rules here. Explicit overrides implicit,
implicit overrides default. Both explicit or both implicit is an error,
etc. Note error state is only a problem if you use it for sorting or
comparison, otherwise it is ignored.

This phase of processing happens in the parse analysis, the end result
being that every expression should have a collation set, and every
operator where it matters has consistant collation information for its
arguments. So at this point the collation should be attached to the
expression.

In the planning phase however, all collation information is ignored
except where it matters, which is the comparison operators and ORDER BY
and similar. By the previous phase all comparison operators should have
a defined collation for its arguments and thus for itself, this allows
it to construct appropriate pathkeys for index scans, etc.

I havn't looked at the patch, perhaps it confuses the information in
the two phases but the basic idea seems to me:

- parse analysis - collation in expressions
- planning - collation in path key

 Or, to put it another way: the properties that define a sort order are
 the sort comparison operator, the collation, the ASC/DESC bit, and the
 NULLS FIRST/LAST bit.  Given the way that the SQL committee has
 constructed the language, the operator and the two flag bits are
 attached to the ORDER BY clause, but the collation is a property of the
 expression-to-be-sorted.  If we fail to preserve that distinction in the
 internal representation, we're just creating problems for ourselves.

The assigning of the collation to expressions is simply the method for
getting the collation information to the operators. Mainly because 99%
of the time there will be no explicit clauses in the queries, so the
information has to get there by other means.

Hope this helps,
-- 
Martijn van Oosterhout   klep...@svana.org   http://svana.org/kleptog/
 Patriotism is when love of your own people comes first; nationalism,
 when hate for people other than your own comes first. 
   - Charles de Gaulle


signature.asc
Description: Digital signature


Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-08 Thread Heikki Linnakangas

On 08.03.2011 10:00, Heikki Linnakangas wrote:

Another idea is to give up on the warning when it appears that
oldestxmin has moved backwards, and assume that it's actually fine. We
could still warn in other cases where the flag appears to be incorrectly
set, like if there is a deleted tuple on the page.


This is probably a better idea at least in back-branches. It also 
handles the case of twiddling vacuum_defer_cleanup_age, which tracking 
two xmins per transactions would not handle.


Here's a patch. I also changed the warning per Robert's suggestion. 
Anyone see a hole in this?


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com
diff --git a/src/backend/commands/vacuumlazy.c b/src/backend/commands/vacuumlazy.c
index 67e0be9..dbd2e8d 100644
--- a/src/backend/commands/vacuumlazy.c
+++ b/src/backend/commands/vacuumlazy.c
@@ -332,6 +332,7 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
 		Size		freespace;
 		bool		all_visible_according_to_vm = false;
 		bool		all_visible;
+		bool		has_dead_tuples;
 
 		/*
 		 * Skip pages that don't require vacuuming according to the visibility
@@ -475,6 +476,7 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
 		 * requiring freezing.
 		 */
 		all_visible = true;
+		has_dead_tuples = false;
 		nfrozen = 0;
 		hastup = false;
 		prev_dead_count = vacrelstats-num_dead_tuples;
@@ -613,6 +615,7 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
 			{
 lazy_record_dead_tuple(vacrelstats, (tuple.t_self));
 tups_vacuumed += 1;
+has_dead_tuples = true;
 			}
 			else
 			{
@@ -671,9 +674,22 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats,
 			PageSetAllVisible(page);
 			SetBufferCommitInfoNeedsSave(buf);
 		}
-		else if (PageIsAllVisible(page)  !all_visible)
+		/*
+		 * It's possible for the value returned by GetOldestXmin() to move
+		 * backwards, so it's not wrong for us to see tuples that appear to
+		 * not be visible to everyone yet, while PD_ALL_VISIBLE is already
+		 * set. The real safe xmin value never moves backwards, but
+		 * GetOldestXmin() is conservative and sometimes returns a value
+		 * that's unnecessarily small, so if we see that contradiction it
+		 * just means that the tuples that we think are not visible to
+		 * everyone yet actually are, and the PD_ALL_VISIBLE flag is correct.
+		 *
+		 * There shouldn't be any dead tuples on a page with PD_ALL_VISIBLE
+		 * set, however.
+		 */
+		else if (PageIsAllVisible(page)  has_dead_tuples)
 		{
-			elog(WARNING, PD_ALL_VISIBLE flag was incorrectly set in relation \%s\ page %u,
+			elog(WARNING, page containing dead tuples is marked as all-visible in relation \%s\ page %u,
  relname, blkno);
 			PageClearAllVisible(page);
 			SetBufferCommitInfoNeedsSave(buf);

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-08 Thread daveg
On Tue, Mar 08, 2011 at 10:00:01AM +0200, Heikki Linnakangas wrote:
 On 08.03.2011 04:07, Greg Stark wrote:
 Well from that log you definitely have OldestXmin going backwards. And
 not by a little bit either. at 6:33 it set the all_visible flag and
 then at 7:01 it was almost 1.3 million transactions earlier. In fact
 to precisely the same value that was in use for a transaction at 1:38.
 That seems like a bit of a coincidence though it's not repeated
 earlier.
 
 Yep. After staring at GetOldestXmin() again, it finally struck me how 
 OldestXmin can move backwards. You need two databases for it, which 
 probably explains why this has been so elusive.
... 
 What there are no other transactions active in the same database, 
 GetOldestXmin() returns just latestCompletedXid. When you open a 
 transaction in the same database after that, its xid will be above 
 latestCompletedXid, but its xmin includes transactions from all 
 databases, and there might be a transaction in some other database with 
 an xid that precedes the value that GetOldestXmin() returned earlier.
 
 I'm not sure what to do about that. One idea is track two xmin values in 
 proc-array, one that includes transactions in all databases, and another 
 that only includes transactions in the same database. GetOldestXmin() 
 (when allDbs is false) would only pay attention to the latter. It would 
 add a few instructions to GetSnapshotData(), though.
 
 Another idea is to give up on the warning when it appears that 
 oldestxmin has moved backwards, and assume that it's actually fine. We 
 could still warn in other cases where the flag appears to be incorrectly 
 set, like if there is a deleted tuple on the page.

I read this to mean that it is safe to ignore this warning and that these
databases are not at risk for data corruption or wrong results so long as
the warning is due to oldestxmin.  Please correct me if I have misunderstood. 

Thanks

-dg

-- 
David Gould   da...@sonic.net  510 536 1443510 282 0869
If simplicity worked, the world would be overrun with insects.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-08 Thread daveg
On Tue, Mar 08, 2011 at 10:37:24AM +0200, Heikki Linnakangas wrote:
 On 08.03.2011 10:00, Heikki Linnakangas wrote:
 Another idea is to give up on the warning when it appears that
 oldestxmin has moved backwards, and assume that it's actually fine. We
 could still warn in other cases where the flag appears to be incorrectly
 set, like if there is a deleted tuple on the page.
 
 This is probably a better idea at least in back-branches. It also 
 handles the case of twiddling vacuum_defer_cleanup_age, which tracking 
 two xmins per transactions would not handle.
 
 Here's a patch. I also changed the warning per Robert's suggestion. 
 Anyone see a hole in this?

It would be helpful to have the dbname and schema in the message in addition
to the relname. I added those to the original diagnostic patch as it was not
clear that the messages were all related to the same page/table/dg.

Also, in your comment you might mention that multiple databases are one way
we could see oldestxmin move backwards.

-dg
 

-- 
David Gould   da...@sonic.net  510 536 1443510 282 0869
If simplicity worked, the world would be overrun with insects.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-08 Thread Heikki Linnakangas

On 08.03.2011 10:38, daveg wrote:

I read this to mean that it is safe to ignore this warning and that these
databases are not at risk for data corruption or wrong results so long as
the warning is due to oldestxmin.  Please correct me if I have misunderstood.


Yes, that's correct.

--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Adrian von Bidder
Heyho!

I'm poking around a bit in PostgreSQL's source for curiosity's sake.

Is there a short howto on how to start / debug postgres from within the 
unpacked/compiled tar? Are there scripts that set up linker paths and 
Postgres' environment?

thanks in advance
-- vbi

-- 
This statement is false.


signature.asc
Description: This is a digitally signed message part.


Re: [HACKERS] pl/python tracebacks

2011-03-08 Thread Jan Urbański
On 07/03/11 22:55, Peter Eisentraut wrote:
 On mån, 2011-03-07 at 14:19 +0100, Jan Urbański wrote:
 On 07/03/11 14:01, Jan Urbański wrote:
 On 07/03/11 13:53, Peter Eisentraut wrote:
 On sön, 2011-03-06 at 13:14 +0100, Jan Urbański wrote:
 But fixing raise plpy.Fatal()
 to actually cause a FATAL is something that should be extracted from
 this patch and committed, even if the full patch does not make it.

 Um, what?  I didn't find any details about this in this thread, nor a
 test case.

 So this in fact are three separate things, tracebacks, fix for
 plpy.Fatal and a one-line fix for reporting errors in Python iterators,
 that as I noticed has a side effect of changing the SQLCODE being raised
 :( I think I'll just respin the tracebacks patch as 3 separate ones,
 coming right up.

 Respun as three separate patches. Sorry for the confusion. BTW: looks
 like plpy.Fatal behaviour has been broken for quite some time now.
 
 Committed 1 and 2.
 
 Your traceback implementation in PLy_elog is now using two errdetail
 calls in one ereport call, which doesn't work (first one wins).  Please
 reconsider that.  Also, the comment still talks about the traceback
 going into detail.

Gah, will look at this and fix.

Jan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: [COMMITTERS] pgsql: Basic Recovery Control functions for use in Hot Standby. Pause,

2011-03-08 Thread Fujii Masao
On Tue, Mar 8, 2011 at 12:04 PM, Fujii Masao masao.fu...@gmail.com wrote:
 On Wed, Feb 9, 2011 at 5:12 PM, Magnus Hagander mag...@hagander.net wrote:
 I was also worried about the non-hot-standby case, but I see that the
 patch makes sure you can't enable pause when not in hot standby mode.
 Which in itself might be surprising - perhaps we need a NOTICE for
 when that happens as well?

 I didn't include this fix in the patch because I prefer FATAL to
 NOTICE for that.
 NOTICE doesn't stop recovery. So we might be unable to notice such a NOTICE
 message and stop the recovery before it's too late, i.e., the recovery has
 completed at the undesirable point. So I think that emitting FATAL is safer.

I included this fix in the patch, which emits FATAL if pause_at_recovery_target
is enabled while hot standby is disabled and the recovery target is set.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


change_recovery_conf_sample_v3.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Sync Rep v19

2011-03-08 Thread Fujii Masao
On Mon, Mar 7, 2011 at 4:54 AM, Robert Haas robertmh...@gmail.com wrote:
 On Mar 6, 2011, at 9:44 AM, Fujii Masao masao.fu...@gmail.com wrote:
 On Sun, Mar 6, 2011 at 5:02 PM, Yeb Havinga yebhavi...@gmail.com wrote:
 On Sun, Mar 6, 2011 at 8:58 AM, Fujii Masao masao.fu...@gmail.com wrote:

 If unfortunately all connection slots are used by backends waiting for
 replication, we cannot execute such a function. So it makes more sense
 to introduce something like pg_ctl standalone command?

 If it is only for shutdown, maybe pg_ctl stop -m standalone?

 It's for not only shutdown but also running the primary in standalone mode.
 So something like pg_ctl standalone is better.

 For now I think that pg_ctl command is better than built-in function because
 sometimes we might want to wake waiters up even during shutdown in
 order to cause shutdown to end. During shutdown, the server doesn't
 accept any new connection (even from the standby). So, without something
 like pg_ctl standalone, there is no way to cause shutdown to end.

 This sounds like an awful hack to work around a bad design. Surely once 
 shutdown reaches a point where new replication connections can no longer be 
 accepted, any standbys hung on commit need to close the connection without 
 responding to the COMMIT, per previous discussion.  It's completely 
 unreasonable for sync rep to break the shutdown sequence.

Yeah, let's think about how shutdown should work. I'd like to propose the
following. Thought?

* Smart shutdown
Smart shutdown should wait for all the waiting backends to be acked, and
should not cause them to forcibly exit. But this leads shutdown to get stuck
infinitely if there is no walsender at that time. To enable them to be acked
even in that situation, we need to change postmaster so that it accepts the
replication connection even during smart shutdown (until we reach
PM_SHUTDOWN_2 state). Postmaster has already accepted the superuser
connection to cancel backup during smart shutdown. So I don't think that
the idea to accept the replication connection during smart shutdown is so
ugly.

* Fast shutdown
I agree with you about fast shutdown. Fast shutdown should cause all the
backends including waiting ones to exit immediately. At that time, the
non-acked backend should not return the success, according to the
definition of sync rep. So we need to change a backend so that it gets rid
of itself from the waiting queue and exits before returning the success,
when it receives SIGTERM. This change leads the waiting backends to
do the same even when pg_terminate_backend is called. But since
they've not been acked yet, it seems to be reasonable to prevent them
from returning the COMMIT.

Comments? I'll create the patch barring objection.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Parallel make problem with git master

2011-03-08 Thread Robert Haas
On Mon, Mar 7, 2011 at 10:28 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 I wrote:
 I think what is happening here is that make launches concurrent sub-jobs
 to do make install in each of interfaces/libpq and interfaces/ecpg,
 and the latter launches a sub-sub-job to do make all in
 interfaces/libpq, and make has no idea that these are duplicate sub-jobs
 so it actually tries to run both concurrently.  Whereupon you get all
 sorts of fun failures.  I'm not sure if there is any cure that's not
 worse than the disease.

 BTW, how many people here have read Recursive Make Considered Harmful?

 http://aegis.sourceforge.net/auug97.pdf

 Because what we're presently doing looks mighty similar to what he's
 saying doesn't work and can't be made to work.

I'm not sure whether it makes sense to go that far or not.  But I
think it'd make sense to at least try this for the backend.  It does
seem pretty silly to have a Makefile in every single directory.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Parallel make problem with git master

2011-03-08 Thread Andrew Dunstan



On 03/07/2011 10:28 PM, Tom Lane wrote:

I wrote:

I think what is happening here is that make launches concurrent sub-jobs
to do make install in each of interfaces/libpq and interfaces/ecpg,
and the latter launches a sub-sub-job to do make all in
interfaces/libpq, and make has no idea that these are duplicate sub-jobs
so it actually tries to run both concurrently.  Whereupon you get all
sorts of fun failures.  I'm not sure if there is any cure that's not
worse than the disease.

BTW, how many people here have read Recursive Make Considered Harmful?

http://aegis.sourceforge.net/auug97.pdf

Because what we're presently doing looks mighty similar to what he's
saying doesn't work and can't be made to work.




Oh, yes, I read it a long time ago, before I started doing Postgres 
work. I recall vaguely thinking about it when I began with Postgres, but 
I thought people smarter than me had probably worked out the problems 
:-) (Working with people smarter than me is one of the things I like 
about Postgres work.)


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Parallel make problem with git master

2011-03-08 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar mar 08 10:38:29 -0300 2011:
 On Mon, Mar 7, 2011 at 10:28 PM, Tom Lane t...@sss.pgh.pa.us wrote:
  I wrote:
  I think what is happening here is that make launches concurrent sub-jobs
  to do make install in each of interfaces/libpq and interfaces/ecpg,
  and the latter launches a sub-sub-job to do make all in
  interfaces/libpq, and make has no idea that these are duplicate sub-jobs
  so it actually tries to run both concurrently.  Whereupon you get all
  sorts of fun failures.  I'm not sure if there is any cure that's not
  worse than the disease.
 
  BTW, how many people here have read Recursive Make Considered Harmful?
 
  http://aegis.sourceforge.net/auug97.pdf
 
  Because what we're presently doing looks mighty similar to what he's
  saying doesn't work and can't be made to work.

Yeah, I read it some years ago and considered it, but it was too
disruptive or I was too new here, maybe both :-)

The bit I looked at, at the time, was src/backend/mb/conversion_procs,
because that was where the biggest hit on parallelization was taken (a
single lib at a time -- the real time CPU usage chart clearly showed the
problem.  Not sure if that's still a problem).

 I'm not sure whether it makes sense to go that far or not.  But I
 think it'd make sense to at least try this for the backend.  It does
 seem pretty silly to have a Makefile in every single directory.

We already do that for the backend.  Not exactly a single Makefile, but
the dependencies are all declared in indirectly in src/backend/Makefile
with the common.mk tricks.

Where it doesn't work is in the other subdirs, c.f. the current problem
with interfaces/libpq and interfaces/ecpg.  It would be a lot more
difficult to fix there, I think, but maybe I'm wrong.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Parallel make problem with git master

2011-03-08 Thread Robert Haas
On Tue, Mar 8, 2011 at 9:07 AM, Alvaro Herrera
alvhe...@commandprompt.com wrote:
 The bit I looked at, at the time, was src/backend/mb/conversion_procs,
 because that was where the biggest hit on parallelization was taken (a
 single lib at a time -- the real time CPU usage chart clearly showed the
 problem.  Not sure if that's still a problem).

I think it is, based on having noticed it spend what seemed like a
disproportionate amount of time on that stuff when building, but I
haven't actually tried to measure it.

 I'm not sure whether it makes sense to go that far or not.  But I
 think it'd make sense to at least try this for the backend.  It does
 seem pretty silly to have a Makefile in every single directory.

 We already do that for the backend.  Not exactly a single Makefile, but
 the dependencies are all declared in indirectly in src/backend/Makefile
 with the common.mk tricks.

I'm not sure that's really the same thing.  It'd be interesting to
redo it with just one Makefile and see whether it's faster.

 Where it doesn't work is in the other subdirs, c.f. the current problem
 with interfaces/libpq and interfaces/ecpg.  It would be a lot more
 difficult to fix there, I think, but maybe I'm wrong.

Yeah, that's a problem.  I wondered if supplying -p to mkdir would
ameliorate the problem to some degree...

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Kevin Grittner
Adrian von Bidder avbid...@fortytwo.ch wrote:
 
 Is there a short howto on how to start / debug postgres from
 within the unpacked/compiled tar? Are there scripts that set up
 linker paths and Postgres' environment?
 
You should probably start by reading the developer FAQ page:
 
http://wiki.postgresql.org/wiki/Developer_FAQ
 
If you still have questions, be sure to mention your OS.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Theory of operation of collation patch

2011-03-08 Thread Susanne Ebrecht

On 07.03.2011 17:43, Tom Lane wrote:

because two expressions that are equal() must necessarily have the same 
collation
property.


Peter, Tom,

I am not able to see this.

If 'abc' == 'abc' is not collation depending at all. It is only
encoding depending.

Collation is only needed for upper(), lower() and sorting.
Means it tells if e.g. upper('i') will get Y or I.
It tells if the German s-umlaut will be sorted together with 's' or 
after 'z'.


Btw. the follows on implementing collations will be different -
and I hope Peter is aware of it.

My experience is that a huge follow will be that users will complain 
that the

sorting isn't correct even when it is correct.

Susanne

--
Susanne Ebrecht - 2ndQuadrant
PostgreSQL Development, 24x7 Support, Training and Services
www.2ndQuadrant.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Parallel make problem with git master

2011-03-08 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 Where it doesn't work is in the other subdirs, c.f. the current problem
 with interfaces/libpq and interfaces/ecpg.  It would be a lot more
 difficult to fix there, I think, but maybe I'm wrong.

Right, it's specifically the interdependence between ecpg and libpq
that's causing the main symptom Jeff is complaining of.  Although when
I was trying make -j12 install starting from a clean tree yesterday,
I did see at least one failure in the backend.  It's all pretty
timing-dependent --- if you look at the make output, you can clearly
see that the same sub-make tasks get launched repeatedly due to various
makefiles trying to force prerequisites in other parts of the tree to be
up to date.  (Which is exactly one of the band-aid fixes that Miller
talks about.)  If two such tasks get launched close enough to the same
time, they both try to do the same work, and then you get failures like
ln complaining that the target is already there, or ar complaining
that somebody corrupted its output file, etc etc.

I think Miller's analysis is dead on and we ought to think seriously
about adopting his approach.  Obviously this is not a small task...

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [DOCS] Sync rep doc corrections

2011-03-08 Thread Thom Brown
On 8 March 2011 01:08, David Fetter da...@fetter.org wrote:
 On Mon, Mar 07, 2011 at 02:31:18PM -0500, Robert Haas wrote:
 On Mon, Mar 7, 2011 at 2:07 PM, David Fetter da...@fetter.org wrote:
  Re: docs, I'd actually like to see that list gone, as the separation
  of docs from code is one that's actively unhelpful.  We don't have a
  separate docs team, and we rightly put the responsibility of
  documenting changes on the person or people patching them in.

 You're in the minority on this one.

 It happens, but I keep speaking up anyhow.  Minority status isn't
 always permanent. :)

Okay, to avoid any -hackers list noise, I'll exclusively send them to
-docs in future, with the exception of replying to -committers or
-hackers when a commit/patch contains fail.

-- 
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] How should the waiting backends behave in sync rep?

2011-03-08 Thread Fujii Masao
Hi,

How should the backends waiting for replication behave when
synchrnous_standby_names
is set to '' and the configuration file is reloaded? Now they keep
waiting for the ACK from the
standby. But I think that it's more natural for them to get out of the
wait state and complete
the transaction in that case. If we'll change them in that way, we
would no longer need
something like pg_ctl standalone which I mentioned in another thread. Thought?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Theory of operation of collation patch

2011-03-08 Thread Tom Lane
Susanne Ebrecht susa...@2ndquadrant.com writes:
 On 07.03.2011 17:43, Tom Lane wrote:
 because two expressions that are equal() must necessarily have the same 
 collation
 property.

 Peter, Tom,

 I am not able to see this.

 If 'abc' == 'abc' is not collation depending at all. It is only
 encoding depending.

Sorry, I was using a term of art there.  This isn't about data values,
it's about parsed expression trees.  See the equal() function in
src/backend/nodes/equalfuncs.c.  If two expression trees match according
to equal(), then they must necessarily have the same collation property,
because equal() compares all the fields and in particular the collation
fields.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Adrian von Bidder
Hi,

On Tuesday 08 March 2011 15.39:56 Kevin Grittner wrote:

 http://wiki.postgresql.org/wiki/Developer_FAQ

thanks

 If you still have questions, be sure to mention your OS.

Sorry, forgot.  Linux.

The Basic system testing item is where I'm kinda stuck. You advise to 
perform run time testing via psql. Which is a bit short, since obviously 
psql can't be directly started since there is no installation...

Aagain, my question: are there wrappers/helpers to do this easily from the 
source/compile tree? Like: set up PATH, LD_LIBRARY_PATH, PG_WHATEVER, call 
initdb, ... 

Or do you advise to just make install and test from there?

thanks
-- vbi

-- 
[...] if you have to do the occasional model for Japanese collectors,
which are covered in reticulated ostrich testicle leather, then by all
means continue to do so.
-- Michael Reichmann
   http://www.luminous-landscape.com/essays/leica-open-letter.shtml


signature.asc
Description: This is a digitally signed message part.


Re: [HACKERS] Theory of operation of collation patch

2011-03-08 Thread Tom Lane
Martijn van Oosterhout klep...@svana.org writes:
 This phase of processing happens in the parse analysis, the end result
 being that every expression should have a collation set, and every
 operator where it matters has consistant collation information for its
 arguments. So at this point the collation should be attached to the
 expression.

Right.

 I havn't looked at the patch, perhaps it confuses the information in
 the two phases but the basic idea seems to me:

 - parse analysis - collation in expressions
 - planning - collation in path key

Not so right.  A path key contains an expression tree, plus whatever
*additional* information is needed to fully specify the sort ordering.
If the collation is already fully determined by the expression tree,
there is no need to duplicate that information in the PathKey node.
And, as I said, doing so anyway has real negative consequences.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Merlin Moncure
On Tue, Mar 8, 2011 at 9:17 AM, Adrian von Bidder avbid...@fortytwo.ch wrote:
 Hi,

 On Tuesday 08 March 2011 15.39:56 Kevin Grittner wrote:

 http://wiki.postgresql.org/wiki/Developer_FAQ

 thanks

 If you still have questions, be sure to mention your OS.

 Sorry, forgot.  Linux.

 The Basic system testing item is where I'm kinda stuck. You advise to
 perform run time testing via psql. Which is a bit short, since obviously
 psql can't be directly started since there is no installation...

 Aagain, my question: are there wrappers/helpers to do this easily from the
 source/compile tree? Like: set up PATH, LD_LIBRARY_PATH, PG_WHATEVER, call
 initdb, ...

sure, linux distos provide packages that do all that stuff. if you are
rolling your own from scratch, you have to do some of that
yourself...create the user, set the path, etc. This is all documented
here: http://www.postgresql.org/docs/9.0/interactive/install-post.html.
 If you are hacking postgres this will become second nature to you
very quickly.

merlin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Kevin Grittner
Adrian von Bidder avbid...@fortytwo.ch wrote:
 
 Or do you advise to just make install and test from there?
 
Pretty much.  For development you will want to specify a few options
in the ./configure step.  At a minimum I recommend:
 
--prefix=
--enable-debug
--enable-cassert
--enable-depend

See this page for other options:
 
http://www.postgresql.org/docs/9.0/interactive/install-procedure.html
 
Of course, there are those who prefer to use eclipse or some other
IDE.  For setting up eclipse, see this page:
 
http://wiki.postgresql.org/wiki/Working_with_Eclipse
 
I see that page is a bit out of date (we've converted from CVS to
git), but perhaps you can extrapolate.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] GSoC 2011 - Mentors? Projects?

2011-03-08 Thread Jehan-Guillaume (ioguix) de Rorthais

On 08/03/2011 07:44, Selena Deckelmann wrote:

Hi!

PostgreSQL is applying for GSoC again this year. We're looking for:

* Mentors


Ready to mentor again this year for phpPgAdmin


* Project ideas

Would you like to mentor? Please let me know! Our application closes
on Friday, so please contact me *before* Friday.

I've started a wiki page: http://wiki.postgresql.org/wiki/GSoC_2011

It's seeded with last year's todo lists and information. We need to
add project ideas for students to it.


I updated the page with last year PPA project


The wiki pages for 2008 and 2010 are available, including links to the
original student proposals:

http://wiki.postgresql.org/wiki/GSoC_2010
http://wiki.postgresql.org/wiki/GSoC_2008

Thanks!
-selena




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] #PgEast Schedule is up

2011-03-08 Thread Joshua D. Drake
Hey,

The schedule for #PgEast is up. It can be found here:

https://www.postgresqlconference.org/files/east_2011_schedule.html

As usually we have a increasingly wide selection of content.

Sincerely,

JD

-- 
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579
Consulting, Training, Support, Custom Development, Engineering
http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Sync Rep v19

2011-03-08 Thread Robert Haas
On Tue, Mar 8, 2011 at 7:05 AM, Fujii Masao masao.fu...@gmail.com wrote:
 Yeah, let's think about how shutdown should work. I'd like to propose the
 following. Thought?

 * Smart shutdown
 Smart shutdown should wait for all the waiting backends to be acked, and
 should not cause them to forcibly exit. But this leads shutdown to get stuck
 infinitely if there is no walsender at that time. To enable them to be acked
 even in that situation, we need to change postmaster so that it accepts the
 replication connection even during smart shutdown (until we reach
 PM_SHUTDOWN_2 state). Postmaster has already accepted the superuser
 connection to cancel backup during smart shutdown. So I don't think that
 the idea to accept the replication connection during smart shutdown is so
 ugly.

 * Fast shutdown
 I agree with you about fast shutdown. Fast shutdown should cause all the
 backends including waiting ones to exit immediately. At that time, the
 non-acked backend should not return the success, according to the
 definition of sync rep. So we need to change a backend so that it gets rid
 of itself from the waiting queue and exits before returning the success,
 when it receives SIGTERM. This change leads the waiting backends to
 do the same even when pg_terminate_backend is called. But since
 they've not been acked yet, it seems to be reasonable to prevent them
 from returning the COMMIT.

The fast shutdown handling seems fine, but why not just handle smart
shutdown the same way?  I don't really like the idea of allowing
replication connections for longer, and the idea that we don't want to
keep waiting for a commit ACK once we're past the point where it's
possible for one to occur seems to apply generically to any shutdown
sequence.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] wrap alpha4 tomorrow ~9am Eastern (was: Alpha4 release blockers)

2011-03-08 Thread Robert Haas
On Mon, Mar 7, 2011 at 5:03 PM, Robert Haas robertmh...@gmail.com wrote:
 On Mon, Mar 7, 2011 at 4:59 PM, Peter Eisentraut pete...@gmx.net wrote:
 On mån, 2011-03-07 at 15:19 -0500, Robert Haas wrote:
 Sorry, you're right.  Still, as happy as I am that we've made so much
 progress with PL/python (and other things) this CommitFest, I think
 it's time to pick a date and time to ship alpha4 and call this one
 good.  If the patch makes it, great; if not, oh well.  We can't keep
 letting this drag out.

 Go for it.

 We might add the traceback patch afterwards, but it's not ready at the
 moment.

 OK, then I propose we tag and cut the tarball on Wednesday morning,
 say around 9am Eastern.

 Objections?

Going once, going twice...

I'll go ahead and do this, barring objections or some other volunteer.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Peter Eisentraut
On tis, 2011-03-08 at 16:17 +0100, Adrian von Bidder wrote:
 Aagain, my question: are there wrappers/helpers to do this easily from
 the source/compile tree? Like: set up PATH, LD_LIBRARY_PATH,
 PG_WHATEVER, call initdb, ... 

Many developers have their own wrapper scripts, mainly to handle dealing
with multiple versions or branches.  But just to build and test one
version you don't need to set any environment variables.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Adrian von Bidder
Hi,

On Tuesday 08 March 2011 16.58:58 Kevin Grittner wrote:
  
 
 Adrian von Bidder avbid...@fortytwo.ch wrote:
  Or do you advise to just make install and test from there?
 Pretty much.

Thanks for all your answers. I was just a bit confused because when I write 
stuff for myself I usually set up stuff to run directly in the build 
environment, so I was expecting something like that to be available.  But 
make install into ~/pg will do fine as well. (I want to avoid having my 
trashed version version be available systemwide...)

cheers
-- vbi

-- 
I've never been DPL (AFAICT)
-- Pierre Habouzit on debian-vote, 2007-07-31


signature.asc
Description: This is a digitally signed message part.


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Tom Lane
Adrian von Bidder avbid...@fortytwo.ch writes:
 Thanks for all your answers. I was just a bit confused because when I write 
 stuff for myself I usually set up stuff to run directly in the build 
 environment, so I was expecting something like that to be available.  But 
 make install into ~/pg will do fine as well. (I want to avoid having my 
 trashed version version be available systemwide...)

No, nobody installs test versions into real locations.  What you want
is to set an install spot with configure --prefix.  For instance, I
usually do
configure --prefix=/home/tgl/testversion --enable-debug --enable-cassert
Be sure to add $PREFIX/bin to your PATH so that you can call the
test-installation programs conveniently.

It's also often a good idea to specify a non-default port number using
--with-pgport so that you can run your test postmaster concurrently with
a regular one.

Lastly, setting PGDATA in your environment to a suitable test database
location will save typing and avoid mistakes.

As Peter mentioned, most of us have scripts to set up a preferred
working environment of this sort.  I think I've published mine at least
once ... check the archives.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote:
 
 As Peter mentioned, most of us have scripts to set up a preferred
 working environment of this sort.  I think I've published mine at
 least once
 
That reminds me -- Greg Smith put something together which might
make it easier to get started:
 
https://github.com/gregs1104/peg/
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Theory of operation of collation patch

2011-03-08 Thread Greg Stark
On Tue, Mar 8, 2011 at 3:21 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Not so right.  A path key contains an expression tree, plus whatever
 *additional* information is needed to fully specify the sort ordering.
 If the collation is already fully determined by the expression tree,
 there is no need to duplicate that information in the PathKey node.
 And, as I said, doing so anyway has real negative consequences.

Isn't the reason to copy that information outside the expression so
that we can choose sometimes to ignore it? Namely, for == we can use
an index with any defined collation even if it doesn't match the
collation in the pathkey we're looking for?

I think currently that's the only example but in theory we could have
collations that are supersets of the desired collation. For example
a UTF8 collation that sorts english in the desired way and sorts utf8
characters in some way that isn't relevant to the query.


-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Theory of operation of collation patch

2011-03-08 Thread Tom Lane
Greg Stark gsst...@mit.edu writes:
 On Tue, Mar 8, 2011 at 3:21 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Not so right.  A path key contains an expression tree, plus whatever
 *additional* information is needed to fully specify the sort ordering.
 If the collation is already fully determined by the expression tree,
 there is no need to duplicate that information in the PathKey node.
 And, as I said, doing so anyway has real negative consequences.

 Isn't the reason to copy that information outside the expression so
 that we can choose sometimes to ignore it? Namely, for == we can use
 an index with any defined collation even if it doesn't match the
 collation in the pathkey we're looking for?

No, that's nonsense.  A PathKey is all about sort order.  It is not
relevant in any situation where sort order is ignorable.

 I think currently that's the only example but in theory we could have
 collations that are supersets of the desired collation. For example
 a UTF8 collation that sorts english in the desired way and sorts utf8
 characters in some way that isn't relevant to the query.

That sounds like nonsense as well.  There is no such thing as a superset
ordering: to have such a thing, you'd have to have some orderings that
didn't fully determine the ordering of data values, which is pretty much
unworkable.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Andrew Dunstan



On 03/08/2011 01:04 PM, Kevin Grittner wrote:

Tom Lanet...@sss.pgh.pa.us  wrote:


As Peter mentioned, most of us have scripts to set up a preferred
working environment of this sort.  I think I've published mine at
least once


That reminds me -- Greg Smith put something together which might
make it easier to get started:

https://github.com/gregs1104/peg/





FWIW, the buildfarm software 
https://github.com/PGBuildFarm/client-code has a couple of developer 
modes. They are particularly useful for testing patches, as the script 
runs through a complete build and test cycle with one command.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-08 Thread Heikki Linnakangas

On 08.03.2011 10:49, daveg wrote:

On Tue, Mar 08, 2011 at 10:37:24AM +0200, Heikki Linnakangas wrote:

On 08.03.2011 10:00, Heikki Linnakangas wrote:

Another idea is to give up on the warning when it appears that
oldestxmin has moved backwards, and assume that it's actually fine. We
could still warn in other cases where the flag appears to be incorrectly
set, like if there is a deleted tuple on the page.


This is probably a better idea at least in back-branches. It also
handles the case of twiddling vacuum_defer_cleanup_age, which tracking
two xmins per transactions would not handle.

Here's a patch. I also changed the warning per Robert's suggestion.
Anyone see a hole in this?


It would be helpful to have the dbname and schema in the message in addition
to the relname. I added those to the original diagnostic patch as it was not
clear that the messages were all related to the same page/table/dg.


Hmm, we don't usually include database name and schema in messages like 
this. There's a couple of other warnings in vacuum, too, that only print 
the table name. I have to admit that the database name was crucial in 
tracking this down, but in hindsight you could've added database name to 
log_line_prefix for the same effect. If you have several databases with 
same schema, that's a good idea anyway. So in the end, I decided not to 
include it.



Also, in your comment you might mention that multiple databases are one way
we could see oldestxmin move backwards.


Ok, I added a comment to GetOldestXmin() explaining that.

Committed. Thanks David for your help in debugging this! And thanks to 
everyone else for helping out too.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: psql include file using relative path

2011-03-08 Thread Ibrar Ahmed
Gurjeet!

What about tab completion, like in \i command?

On Fri, Feb 25, 2011 at 5:07 AM, Robert Haas robertmh...@gmail.com wrote:
 On Thu, Feb 24, 2011 at 6:21 PM, Gurjeet Singh singh.gurj...@gmail.com 
 wrote:
     psql has the ability to execute commands from a file, but if one wishes
 to develop and provide a modularized set of sql files, then psql is not very
 helpful because the \i command can open file paths either if they are
 absolute paths or if they are palced correctly relative to psql's current
 working directory.

 Attached patch adds a new meta-command to psql, '\ir' that allows the user
 to process files relative to currently processing file.

 Please add this patch to the currently open CommitFest at:

 https://commitfest.postgresql.org/action/commitfest_view/open

 --
 Robert Haas
 EnterpriseDB: http://www.enterprisedb.com
 The Enterprise PostgreSQL Company

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers




-- 
   Ibrar Ahmed

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] .gitignore patch for coverage builds

2011-03-08 Thread Peter Eisentraut
On ons, 2011-01-26 at 15:38 -0600, Kevin Grittner wrote:
 Kevin Grittner kevin.gritt...@wicourts.gov wrote:
  
  Patch attached.
  
 The coverage directory belongs under Local excludes in root
 directory.  Version 2.

I have committed a simplified version of this, except the coverage/
directory, which some people had issues with.  I think it's OK to
manually ignore that.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Adrian von Bidder
Heyho again!

Now this kind of stuff is what I was after :-)

On Tuesday 08 March 2011 19.04:53 Kevin Grittner wrote:
 That reminds me -- Greg Smith put something together which might
 make it easier to get started:
  
 https://github.com/gregs1104/peg/

Since you don't allow anonymous editing of the wiki, nor have an obvious 
create account, I leave it to you to add this:

+++
How can I easily test my modified postgres version?

You'll want to install PostgreSQL into a local directory (in your home 
directory, for instance) to avoid conflicting with a system wide 
installation.  Use the --prefix=... option to specify an installation 
location; --with-pgport to specify a non-standard default port may also be 
helpful. To run this instance, you will need to make sure that the correct 
binaries are used; depending on your operating system, environment variables 
like PATH and LD_LIBRARY_PATH (on most Linux/Unix-like systems) need to be 
set. Setting PGDATA will also be useful.

To avoid having to set this environment up manually, you may want to use 
Greg Smith's [href=https://github.com/gregs1104/peg peg] scripts, or the 
[https://github.com/PGBuildFarm/client-code scripts] that are used on the 
buildfarm.
+++

... and you'll never have to answer to this stupid question again ... :-)

thanks  happy hacking
-- vbi

P.S.: those who remember me from many months back may guess that I'm trying 
to implement IMMUTABLE columns.  But I'll come back when I actually have to 
show something, so far I've only managed to break 121 of 124 tests of make 
check. At least initdb miraculously works again.

-- 
this email is protected by a digital signature: http://fortytwo.ch/gpg


signature.asc
Description: This is a digitally signed message part.


Re: [HACKERS] GSoC 2011 - Mentors? Projects?

2011-03-08 Thread Alexander Korotkov
Hello!

I've following project proposal.

Currently GiST index don't have any bulk load functionality. It have to
create new index by entry insertion one by one. This makes new index
creation relatively slow.
There are various works in computer science about bulk operation on R-tree.
Since gist in basic is generalization of R-tree, some of them seems to
be applicable to GiST.
Following work seems to be most applicable to GiST:
Efficient Bulk Operations on Dynamic R-Trees, L. Arge, K. H. Hinrichs, J.
Vahrenhold and J. S. Vitter
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.135.9894rep=rep1type=pdf
I believe that implementation of methods of this work should allow to
significally reduce GiST index creation time (probably, it'll be possible to
achive acceleration of other operations too). Also, it's possible to achive
some results in computer science, since GiST is more general data structure
than R-tree.
I'm going to create wiki page for this project proposal soon.

I'm last year phd. student and I would like to participate in GSoC with
PostgreSQL. I would be especially happy to work with project above or other
project related to indexing.


With best regards,
Alexander Korotkov.

On Tue, Mar 8, 2011 at 9:44 AM, Selena Deckelmann sel...@chesnok.comwrote:

 Hi!

 PostgreSQL is applying for GSoC again this year. We're looking for:

 * Mentors
 * Project ideas

 Would you like to mentor? Please let me know! Our application closes
 on Friday, so please contact me *before* Friday.

 I've started a wiki page: http://wiki.postgresql.org/wiki/GSoC_2011

 It's seeded with last year's todo lists and information. We need to
 add project ideas for students to it.

 The wiki pages for 2008 and 2010 are available, including links to the
 original student proposals:

 http://wiki.postgresql.org/wiki/GSoC_2010
 http://wiki.postgresql.org/wiki/GSoC_2008

 Thanks!
 -selena

 --
 http://chesnok.com

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers



Re: [HACKERS] Fwd: psql include file using relative path

2011-03-08 Thread Gurjeet Singh
Attached patch implements tab completion. It also introduces the long-form
alternative \include_relative for \ir

Regards,

On Tue, Mar 8, 2011 at 2:18 PM, Ibrar Ahmed ibrar.ah...@gmail.com wrote:

 Gurjeet!

 What about tab completion, like in \i command?

 On Fri, Feb 25, 2011 at 5:07 AM, Robert Haas robertmh...@gmail.com
 wrote:
  On Thu, Feb 24, 2011 at 6:21 PM, Gurjeet Singh singh.gurj...@gmail.com
 wrote:
  psql has the ability to execute commands from a file, but if one
 wishes
  to develop and provide a modularized set of sql files, then psql is not
 very
  helpful because the \i command can open file paths either if they are
  absolute paths or if they are palced correctly relative to psql's
 current
  working directory.
 
  Attached patch adds a new meta-command to psql, '\ir' that allows the
 user
  to process files relative to currently processing file.
 
  Please add this patch to the currently open CommitFest at:
 
  https://commitfest.postgresql.org/action/commitfest_view/open
 
  --
  Robert Haas
  EnterpriseDB: http://www.enterprisedb.com
  The Enterprise PostgreSQL Company
 
  --
  Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
  To make changes to your subscription:
  http://www.postgresql.org/mailpref/pgsql-hackers
 



 --
Ibrar Ahmed




-- 
Gurjeet Singh
EnterpriseDB http://www.enterprisedb.com/ Corporation
The Enterprise PostgreSQL http://www.postgresql.org/ Company


psql_ir.patch.gz
Description: GNU Zip compressed data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-08 Thread Andres Freund
Hi,

On Tuesday 08 March 2011 21:12:53 Adrian von Bidder wrote:
 Since you don't allow anonymous editing of the wiki, nor have an obvious
 create account, I leave it to you to add this:
Its linked on the mainpage: http://www.postgresql.org/community/signup

Andres

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] GSoC 2011 - Mentors? Projects?

2011-03-08 Thread Oleg Bartunov

Yes,

this would be very interesting project. It's not just about 
bulk load (CREATE INDEX) but also about bulk insert, which is important

for update operations. As it adds two  additional internal GiST methods, there
should be no problem with compatibility.

Oleg

On Tue, 8 Mar 2011, Alexander Korotkov wrote:


Hello!

I've following project proposal.

Currently GiST index don't have any bulk load functionality. It have to
create new index by entry insertion one by one. This makes new index
creation relatively slow.
There are various works in computer science about bulk operation on R-tree.
Since gist in basic is generalization of R-tree, some of them seems to
be applicable to GiST.
Following work seems to be most applicable to GiST:
Efficient Bulk Operations on Dynamic R-Trees, L. Arge, K. H. Hinrichs, J.
Vahrenhold and J. S. Vitter
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.135.9894rep=rep1type=pdf
I believe that implementation of methods of this work should allow to
significally reduce GiST index creation time (probably, it'll be possible to
achive acceleration of other operations too). Also, it's possible to achive
some results in computer science, since GiST is more general data structure
than R-tree.
I'm going to create wiki page for this project proposal soon.

I'm last year phd. student and I would like to participate in GSoC with
PostgreSQL. I would be especially happy to work with project above or other
project related to indexing.


With best regards,
Alexander Korotkov.

On Tue, Mar 8, 2011 at 9:44 AM, Selena Deckelmann sel...@chesnok.comwrote:


Hi!

PostgreSQL is applying for GSoC again this year. We're looking for:

* Mentors
* Project ideas

Would you like to mentor? Please let me know! Our application closes
on Friday, so please contact me *before* Friday.

I've started a wiki page: http://wiki.postgresql.org/wiki/GSoC_2011

It's seeded with last year's todo lists and information. We need to
add project ideas for students to it.

The wiki pages for 2008 and 2010 are available, including links to the
original student proposals:

http://wiki.postgresql.org/wiki/GSoC_2010
http://wiki.postgresql.org/wiki/GSoC_2008

Thanks!
-selena

--
http://chesnok.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers





Regards,
Oleg
_
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Parallel make problem with git master

2011-03-08 Thread Peter Eisentraut
On mån, 2011-03-07 at 13:51 -0800, Jeff Davis wrote:
 On Sat, 2011-03-05 at 18:33 -0500, Bruce Momjian wrote:
  I am seeing the following compile problem with gmake -j2:
  
 
 For what it's worth, I'm still seeing this problem too:
 
 http://archives.postgresql.org/pgsql-hackers/2010-12/msg00123.php
 
 I can reproduce it every time.

Fixed.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Parallel make problem with git master

2011-03-08 Thread Peter Eisentraut
On mån, 2011-03-07 at 22:28 -0500, Tom Lane wrote:
 BTW, how many people here have read Recursive Make Considered
 Harmful?
 
 http://aegis.sourceforge.net/auug97.pdf
 
 Because what we're presently doing looks mighty similar to what he's
 saying doesn't work and can't be made to work.

Yes, that's the better solution.  It will probably just upset a lot of
people's thinking.

The main problem way back when I last considered this seriously was that
it wasn't clear how many compilers don't support -o with -c.  The paper
doesn't offer a clear solution to that, but it might be that the problem
is effectively gone now.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)

2011-03-08 Thread Peter Eisentraut
On mån, 2011-03-07 at 12:40 -0500, Tom Lane wrote:
 Fair enough, but throwing in fmgr_info_collation(DEFAULT_COLLATION)
 anytime we have a problem seems to me to introduce the exact same
 issue.  Who's to say that that's really the appropriate value to use?

It normally isn't the appropriate value to use.  It's only appropriate
if either that particular part of the code doesn't support collations
yet or it's dealing with some hardcoded catalog lookups or some similar
case.  In most other cases you should be getting the collation passed in
from somewhere, and if you aren't there is probably some work to do to
get it there.  That's at least sort of the experience from developing
this.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)

2011-03-08 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 On mån, 2011-03-07 at 12:40 -0500, Tom Lane wrote:
 Fair enough, but throwing in fmgr_info_collation(DEFAULT_COLLATION)
 anytime we have a problem seems to me to introduce the exact same
 issue.  Who's to say that that's really the appropriate value to use?

 It normally isn't the appropriate value to use.  It's only appropriate
 if either that particular part of the code doesn't support collations
 yet or it's dealing with some hardcoded catalog lookups or some similar
 case.  In most other cases you should be getting the collation passed in
 from somewhere, and if you aren't there is probably some work to do to
 get it there.  That's at least sort of the experience from developing
 this.

Mph.  Well, I guess in the case of the pg_statistic stats we can declare
by fiat that we calculate the stats according to the default collation.
They'll be a bit off when used for a query that is comparing according
to some other collation, but it's probably no worse than any other
approximation we make in the selectivity code.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Theory of operation of collation patch

2011-03-08 Thread Peter Eisentraut
On mån, 2011-03-07 at 12:52 -0500, Tom Lane wrote:
 It looks like indcollation is acting as a
 substitute for including a CollateClause in the index key expression,
 which doesn't seem like a particularly good tradeoff considering all
 the overhead you must introduce into the default case.

Conceptually, this characterization is correct.  But assuming you do
away with the indcollation column and instead create expression indexes
on (col COLLATE x), you'd then have an interesting time matching those
expressions to expressions in the query when choosing an index.
Example:

CREATE TABLE test (a text);
CREATE INDEX test_idx ((a COLLATE foo));  -- expression index

-- easy to choose the index here
SELECT * FROM test WHERE a COLLATE foo  'x';

-- not so easy here
SELECT * FROM test WHERE a  'x' COLLATE foo;

-- more difficult
CREATE TABLE test2 (b text COLLATE foo);
SELECT * FROM test WHERE x  (SELECT b FROM test2 WHERE something);

Note that this situation already exists: If you create a plain column
index using CREATE INDEX test_idx (a COLLATE foo), all of the above
cases should pick up the index; if you create an expression index using
CREATE INDEX test_idx ((a COLLATE foo)), they won't, except the first
one.  (That is modulo your earlier demonstration that it apparently
doesn't work at all for you, but I'm just saying in case you want to try
out various ways of changing this.)



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Theory of operation of collation patch

2011-03-08 Thread Peter Eisentraut
On mån, 2011-03-07 at 19:15 -0500, Tom Lane wrote:
 Or, to put it another way: the properties that define a sort order are
 the sort comparison operator, the collation, the ASC/DESC bit, and the
 NULLS FIRST/LAST bit.  Given the way that the SQL committee has
 constructed the language, the operator and the two flag bits are
 attached to the ORDER BY clause, but the collation is a property of
 the expression-to-be-sorted.  If we fail to preserve that distinction
 in the internal representation, we're just creating problems for
 ourselves.

You are probably right; it could be simplified.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha4 release blockers (was Re: wrapping up this CommitFest)

2011-03-08 Thread Peter Eisentraut
On tis, 2011-03-08 at 17:43 -0500, Tom Lane wrote:
 Mph.  Well, I guess in the case of the pg_statistic stats we can
 declare by fiat that we calculate the stats according to the default
 collation.  They'll be a bit off when used for a query that is
 comparing according to some other collation, but it's probably no
 worse than any other approximation we make in the selectivity code.

That was my assumption for the time being.

Unfortunately, it's hard to create test coverage for these types of
issues.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Theory of operation of collation patch

2011-03-08 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 On mån, 2011-03-07 at 12:52 -0500, Tom Lane wrote:
 It looks like indcollation is acting as a
 substitute for including a CollateClause in the index key expression,
 which doesn't seem like a particularly good tradeoff considering all
 the overhead you must introduce into the default case.

 Conceptually, this characterization is correct.  But assuming you do
 away with the indcollation column and instead create expression indexes
 on (col COLLATE x), you'd then have an interesting time matching those
 expressions to expressions in the query when choosing an index.

Hmm ... interesting point, but I think it already doesn't work in all
cases for expression indexes.  There is an analogous issue for
binary-compatible datatypes, which we hack around by stripping
RelabelType nodes before doing this type of comparison.  Maybe we'll
have to do something similar with CollateClauses.  I never much cared
for that hack, though ... wonder if there is a cleaner way.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] GSoC 2011 - Mentors? Projects?

2011-03-08 Thread Selena Deckelmann
Hi Alexander,

On Tue, Mar 8, 2011 at 12:13 PM, Alexander Korotkov
aekorot...@gmail.com wrote:

 I've following project proposal.

Thanks for getting a head start! One warning - we haven't been
accepted into GSoC yet. :) I'll be sure to contact you if we are
accepted!

 Currently GiST index don't have any bulk load functionality. It have to
 create new index by entry insertion one by one. This makes new index
 creation relatively slow.
 There are various works in computer science about bulk operation on R-tree.
 Since gist in basic is generalization of R-tree, some of them seems to
 be applicable to GiST.
 Following work seems to be most applicable to GiST:
 Efficient Bulk Operations on Dynamic R-Trees, L. Arge, K. H. Hinrichs, J.
 Vahrenhold and J. S. Vitter
 http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.135.9894rep=rep1type=pdf
 I believe that implementation of methods of this work should allow to
 significally reduce GiST index creation time (probably, it'll be possible to
 achive acceleration of other operations too). Also, it's possible to achive
 some results in computer science, since GiST is more general data structure
 than R-tree.
 I'm going to create wiki page for this project proposal soon.

That sounds great. Please go ahead and create the page now.

-selena


-- 
http://chesnok.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] GSoC 2011 - Mentors? Projects?

2011-03-08 Thread Tatsuo Ishii
 Hi!
 
 PostgreSQL is applying for GSoC again this year. We're looking for:
 
 * Mentors

Can I be a metor for pgpool-II?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

 * Project ideas
 
 Would you like to mentor? Please let me know! Our application closes
 on Friday, so please contact me *before* Friday.
 
 I've started a wiki page: http://wiki.postgresql.org/wiki/GSoC_2011
 
 It's seeded with last year's todo lists and information. We need to
 add project ideas for students to it.
 
 The wiki pages for 2008 and 2010 are available, including links to the
 original student proposals:
 
 http://wiki.postgresql.org/wiki/GSoC_2010
 http://wiki.postgresql.org/wiki/GSoC_2008
 
 Thanks!
 -selena
 
 -- 
 http://chesnok.com
 
 -- 
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] GSoC 2011 - Mentors? Projects?

2011-03-08 Thread Selena Deckelmann
On Tue, Mar 8, 2011 at 3:52 PM, Tatsuo Ishii is...@postgresql.org wrote:
 Hi!

 PostgreSQL is applying for GSoC again this year. We're looking for:

 * Mentors

 Can I be a metor for pgpool-II?

Yes! I can't guarantee that we will get student proposals for it. But,
we will have a greater chance of student proposals for things if we
specifically list out projects that you'd like a student to tackle.

Could you add a few items to the wiki page for pgpool-II?
http://wiki.postgresql.org/wiki/GSoC_2011#Project_Ideas

Thanks!
-selena


-- 
http://chesnok.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] GSoC 2011 - Mentors? Projects?

2011-03-08 Thread Tatsuo Ishii
 Can I be a metor for pgpool-II?
 
 Yes! I can't guarantee that we will get student proposals for it. But,
 we will have a greater chance of student proposals for things if we
 specifically list out projects that you'd like a student to tackle.
 
 Could you add a few items to the wiki page for pgpool-II?
 http://wiki.postgresql.org/wiki/GSoC_2011#Project_Ideas

Ok, I will do it.

 Thanks!

Thanks!
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Theory of operation of collation patch

2011-03-08 Thread Tom Lane
Another interesting item ... I see that you added a collation field to
TypeName, apparently on the grounds that the SQL spec includes collation
in data type.  However, it seems to me that that is nonsense up with
which we should not put.  data type is basically only used in CAST and
column definitions (plus some features related to methods, which we
haven't got).  In CAST, they have to have a syntax rule stating that you
must not use COLLATE in the target type name.  In column definitions,
they call out collate clause as a separate syntactic element, and then
they have to have a syntax rule stating that you can't use both that
and a collate clause embedded in the type name.  This is just stupid.
And it conflates the notion of collation with that of type --- based on
our discussions so far, it seems like it'd be a lot better to keep those
separate.

I think we should drop collate clause from TypeName and just have it
in columnDef and the expression syntax.  This might also ease the
ambiguity problem that evidently led you to restrict the expression
production's argument to c_expr.  It would also allow us to meet the
letter of the spec for column definition, in that collate clause
is not required to immediately follow data type.  I see that right now
you don't allow collate clause in ColQualList, so it's not accepting
syntax that the spec says is valid; and you really can't fix that as long
as collate clause can be part of TypeName, because then there would be
two ways to parse the case where COLLATE does immediately follow the
type name.

Thoughts?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Fix for pg_upgrade discriptor leaks

2011-03-08 Thread Bruce Momjian
I have applied the attached patch to fix pg_upgrade file descriptor
leaks in error paths.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +
diff --git a/contrib/pg_upgrade/check.c b/contrib/pg_upgrade/check.c
index 9ef63ec..1c4847a 100644
*** a/contrib/pg_upgrade/check.c
--- b/contrib/pg_upgrade/check.c
*** check_for_isn_and_int8_passing_mismatch(
*** 515,523 
  		PQfinish(conn);
  	}
  
  	if (found)
  	{
- 		fclose(script);
  		pg_log(PG_REPORT, fatal\n);
  		pg_log(PG_FATAL,
  			   | Your installation contains \/contrib/isn\ functions\n
--- 515,525 
  		PQfinish(conn);
  	}
  
+ 	if (script)
+ 			fclose(script);
+ 
  	if (found)
  	{
  		pg_log(PG_REPORT, fatal\n);
  		pg_log(PG_FATAL,
  			   | Your installation contains \/contrib/isn\ functions\n
*** check_for_reg_data_type_usage(ClusterInf
*** 616,624 
  		PQfinish(conn);
  	}
  
  	if (found)
  	{
- 		fclose(script);
  		pg_log(PG_REPORT, fatal\n);
  		pg_log(PG_FATAL,
  			   | Your installation contains one of the reg* data types in\n
--- 618,628 
  		PQfinish(conn);
  	}
  
+ 	if (script)
+ 		fclose(script);
+ 
  	if (found)
  	{
  		pg_log(PG_REPORT, fatal\n);
  		pg_log(PG_FATAL,
  			   | Your installation contains one of the reg* data types in\n
diff --git a/contrib/pg_upgrade/file.c b/contrib/pg_upgrade/file.c
index deaca46..803e0a0 100644
*** a/contrib/pg_upgrade/file.c
--- b/contrib/pg_upgrade/file.c
*** pg_scandir_internal(const char *dirname,
*** 302,308 
--- 302,311 
  		(size_t) ((name_num + 1) * sizeof(struct dirent *)));
  
  			if (*namelist == NULL)
+ 			{
+ closedir(dirdesc);
  return -1;
+ 			}
  
  			entrysize = sizeof(struct dirent) - sizeof(direntry-d_name) +
  strlen(direntry-d_name) + 1;
*** pg_scandir_internal(const char *dirname,
*** 310,316 
--- 313,322 
  			(*namelist)[name_num] = (struct dirent *) malloc(entrysize);
  
  			if ((*namelist)[name_num] == NULL)
+ 			{
+ closedir(dirdesc);
  return -1;
+ 			}
  
  			memcpy((*namelist)[name_num], direntry, entrysize);
  
diff --git a/contrib/pg_upgrade/server.c b/contrib/pg_upgrade/server.c
index c7684f8..84bd03e 100644
*** a/contrib/pg_upgrade/server.c
--- b/contrib/pg_upgrade/server.c
*** get_major_server_version(ClusterInfo *cl
*** 144,154 
  	if (fscanf(version_fd, %63s, cluster-major_version_str) == 0 ||
  		sscanf(cluster-major_version_str, %d.%d, integer_version,
  			   fractional_version) != 2)
- 	{
  		pg_log(PG_FATAL, could not get version from %s\n, datadir);
! 		fclose(version_fd);
! 		return 0;
! 	}
  
  	return (100 * integer_version + fractional_version) * 100;
  }
--- 144,152 
  	if (fscanf(version_fd, %63s, cluster-major_version_str) == 0 ||
  		sscanf(cluster-major_version_str, %d.%d, integer_version,
  			   fractional_version) != 2)
  		pg_log(PG_FATAL, could not get version from %s\n, datadir);
! 
! 	fclose(version_fd);
  
  	return (100 * integer_version + fractional_version) * 100;
  }
diff --git a/contrib/pg_upgrade/version.c b/contrib/pg_upgrade/version.c
index e321538..8ba7e98 100644
*** a/contrib/pg_upgrade/version.c
--- b/contrib/pg_upgrade/version.c
*** new_9_0_populate_pg_largeobject_metadata
*** 62,71 
  		PQfinish(conn);
  	}
  
  	if (found)
  	{
- 		if (!check_mode)
- 			fclose(script);
  		report_status(PG_WARNING, warning);
  		if (check_mode)
  			pg_log(PG_WARNING, \n
--- 62,72 
  		PQfinish(conn);
  	}
  
+ 	if (script)
+ 		fclose(script);
+ 
  	if (found)
  	{
  		report_status(PG_WARNING, warning);
  		if (check_mode)
  			pg_log(PG_WARNING, \n
diff --git a/contrib/pg_upgrade/version_old_8_3.c b/contrib/pg_upgrade/version_old_8_3.c
index 5b226b2..3ec4b59 100644
*** a/contrib/pg_upgrade/version_old_8_3.c
--- b/contrib/pg_upgrade/version_old_8_3.c
*** old_8_3_check_for_name_data_type_usage(C
*** 87,95 
  		PQfinish(conn);
  	}
  
  	if (found)
  	{
- 		fclose(script);
  		pg_log(PG_REPORT, fatal\n);
  		pg_log(PG_FATAL,
  			   | Your installation contains the \name\ data type in\n
--- 87,97 
  		PQfinish(conn);
  	}
  
+ 	if (script)
+ 		fclose(script);
+ 
  	if (found)
  	{
  		pg_log(PG_REPORT, fatal\n);
  		pg_log(PG_FATAL,
  			   | Your installation contains the \name\ data type in\n
*** old_8_3_check_for_tsquery_usage(ClusterI
*** 175,183 
  		PQfinish(conn);
  	}
  
  	if (found)
  	{
- 		fclose(script);
  		pg_log(PG_REPORT, fatal\n);
  		pg_log(PG_FATAL,
  			   | Your installation contains the \tsquery\ data type.\n
--- 177,187 
  		PQfinish(conn);
  	}
  
+ 	if (script)
+ 		fclose(script);
+ 
  	if (found)
  	{
  		pg_log(PG_REPORT, fatal\n);
  		pg_log(PG_FATAL,
  			   | Your installation contains the \tsquery\ data type.\n
*** old_8_3_rebuild_tsvector_tables(ClusterI
*** 314,323 
  		

Re: [HACKERS] Fix for pg_upgrade discriptor leaks

2011-03-08 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 I have applied the attached patch to fix pg_upgrade file descriptor
 leaks in error paths.

It seems rather pointless to spend code closing descriptors immediately
before a fatal exit.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fix for pg_upgrade discriptor leaks

2011-03-08 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian br...@momjian.us writes:
  I have applied the attached patch to fix pg_upgrade file descriptor
  leaks in error paths.
 
 It seems rather pointless to spend code closing descriptors immediately
 before a fatal exit.

Well, it is not before a fatal but rather before it returns -1, which
might fatal or might not.  I also had code in to close file descriptors
that was a little too tricky about using a single variable to indicate
two things so I cleaned it up.  I got a private email report about these
so obviously they were confusing.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Sync Rep v19

2011-03-08 Thread Jaime Casanova
On Tue, Mar 8, 2011 at 11:58 AM, Robert Haas robertmh...@gmail.com wrote:

 The fast shutdown handling seems fine, but why not just handle smart
 shutdown the same way?

currently, smart shutdown means no new connections, wait until
existing ones close normally. for consistency, it should behave the
same for sync rep.

+1 for Fujii's proposal

-- 
Jaime Casanova         www.2ndQuadrant.com
Professional PostgreSQL: Soporte y capacitación de PostgreSQL

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-08 Thread Fujii Masao
On Wed, Mar 9, 2011 at 2:14 PM, Jaime Casanova ja...@2ndquadrant.com wrote:
 On Tue, Mar 8, 2011 at 11:58 AM, Robert Haas robertmh...@gmail.com wrote:

 The fast shutdown handling seems fine, but why not just handle smart
 shutdown the same way?

 currently, smart shutdown means no new connections, wait until
 existing ones close normally. for consistency, it should behave the
 same for sync rep.

Agreed. I think that user who wants to request smart shutdown expects all
the existing connections to basically be closed normally by the client. So it
doesn't seem to be good idea to forcibly close the connection and prevent
the COMMIT from being returned in smart shutdown case. But I'm all ears
for better suggestions.

Anyway, we got the consensus about how fast shutdown should work with
sync rep. So I created the patch. Please feel free to comment and commit
the patch first ;)

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


fast_shutdown_syncrep_v1.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers