Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-09 Thread Yeb Havinga

On 2011-03-09 08:38, Fujii Masao wrote:

On Wed, Mar 9, 2011 at 2:14 PM, Jaime Casanovaja...@2ndquadrant.com  wrote:

On Tue, Mar 8, 2011 at 11:58 AM, Robert Haasrobertmh...@gmail.com  wrote:

The fast shutdown handling seems fine, but why not just handle smart
shutdown the same way?

currently, smart shutdown means no new connections, wait until
existing ones close normally. for consistency, it should behave the
same for sync rep.

Agreed. I think that user who wants to request smart shutdown expects all
the existing connections to basically be closed normally by the client. So it
doesn't seem to be good idea to forcibly close the connection and prevent
the COMMIT from being returned in smart shutdown case. But I'm all ears
for better suggestions.
For me smart has always been synonymous to no forced disconnects/exits, 
or put different, the 'clean' solution, as opposite to the fast and 
unclean shutdown.


An alternative for a clean solution might be to forbid smart shutdown, 
if none of the sync standbys is connected. This would prevent the master 
to enter a state in which a standby cannot connect anymore.


regards,
Yeb Havinga


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] What happens If a table changes during a query/procedure execution

2011-03-09 Thread Vlad Arkhipov
Let there are two transactions that were created with read commited 
isolation level. In the first one we're executing a SELECT query:

SELECT * FROM t UNION ALL SELECT * FROM t;

In the second transaction we're modifying the same table:
INSERT INTO t DEFAULT VALUES;
COMMIT;

Is it possible that the last UNION part in the first query will retrieve 
not the same rows as the first one?
Another scenario is where we're executing two SELECT queries in a stored 
procedure:

BEGIN
  ...
  SELECT * FROM t;
  SELECT * FROM t;
END;

Is it possible to get different results in the second query? Does SQL 
standard define the behaviour in such cases?


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] What happens If a table changes during a query/procedure execution

2011-03-09 Thread Nicolas Barbier
2011/3/9 Vlad Arkhipov arhi...@dc.baikal.ru:

 Let there are two transactions that were created with read commited
 isolation level. In the first one we're executing a SELECT query:
 SELECT * FROM t UNION ALL SELECT * FROM t;

 In the second transaction we're modifying the same table:
 INSERT INTO t DEFAULT VALUES;
 COMMIT;

 Is it possible that the last UNION part in the first query will retrieve not
 the same rows as the first one?

No, because statements never see changes made by other transactions
while they are in flight.

 Another scenario is where we're executing two SELECT queries in a stored
 procedure:
 BEGIN
  ...
  SELECT * FROM t;
  SELECT * FROM t;
 END;

 Is it possible to get different results in the second query?

Yes, because they are separate statements, and in READ COMMITTED mode,
a new snapshot is taken when a statement starts. See:
URL:http://www.postgresql.org/docs/9.0/static/transaction-iso.html#XACT-READ-COMMITTED.

 Does SQL standard define the behaviour in such cases?

The first one certainly. The standard doesn't describe PL/PgSQL, so
the question is moot in the second case; nonetheless, I assume that
the answer would be yes in the case of SQL/PSM.

Note that the standard defines things that must never happen in the
case of READ COMMITTED, it does not specify that one *must* be able to
see the stuff as committed by previous transactions, for example.

Nicolas

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] What happens If a table changes during a query/procedure execution

2011-03-09 Thread Nicolas Barbier
2011/3/9 Nicolas Barbier nicolas.barb...@gmail.com:

 Note that the standard defines things that must never happen in the
 case of READ COMMITTED, it does not specify that one *must* be able to
 see the stuff as committed by previous transactions, for example.

Hmm, make that stuff as committed by concurrent transactions that
committed prior to our statement's execution.

Nicolas

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Update of replication/README

2011-03-09 Thread Fujii Masao
Hi,

The attached patch updates replication/README to reflect current
walsender/walreceiver behavior. It doesn't include any description
about sync rep. That would need to be added later.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


replication_readme_v1.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] What happens If a table changes during a query/procedure execution

2011-03-09 Thread Vlad Arkhipov

09.03.2011 18:54, Nicolas Barbier:

2011/3/9 Vlad Arkhipovarhi...@dc.baikal.ru:

   

Let there are two transactions that were created with read commited
isolation level. In the first one we're executing a SELECT query:
SELECT * FROM t UNION ALL SELECT * FROM t;

In the second transaction we're modifying the same table:
INSERT INTO t DEFAULT VALUES;
COMMIT;

Is it possible that the last UNION part in the first query will retrieve not
the same rows as the first one?
 

No, because statements never see changes made by other transactions
while they are in flight.

   
Is it also true if a statement contains subqueries or function calls? 
For instance,


CREATE FUNCTION f() RETURNS NUMERIC AS $$
BEGIN
  RETURN (SELECT SUM(a) FROM t);
END;
$$ LANGUAGE 'plpgsql';

SELECT a, f() FROM t;

or

SELECT a, (SELECT SUM(a) FROM t) FROM t;


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] What happens If a table changes during a query/procedure execution

2011-03-09 Thread Pavel Stehule
2011/3/9 Vlad Arkhipov arhi...@dc.baikal.ru:
 09.03.2011 18:54, Nicolas Barbier:

 2011/3/9 Vlad Arkhipovarhi...@dc.baikal.ru:



 Let there are two transactions that were created with read commited
 isolation level. In the first one we're executing a SELECT query:
 SELECT * FROM t UNION ALL SELECT * FROM t;

 In the second transaction we're modifying the same table:
 INSERT INTO t DEFAULT VALUES;
 COMMIT;

 Is it possible that the last UNION part in the first query will retrieve
 not
 the same rows as the first one?


 No, because statements never see changes made by other transactions
 while they are in flight.



 Is it also true if a statement contains subqueries or function calls? For
 instance,

 CREATE FUNCTION f() RETURNS NUMERIC AS $$
 BEGIN
  RETURN (SELECT SUM(a) FROM t);
 END;
 $$ LANGUAGE 'plpgsql';

 SELECT a, f() FROM t;

 or

 SELECT a, (SELECT SUM(a) FROM t) FROM t;

yes, it is same

Regards

Pavel Stehule



 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] pl/python - thanks!

2011-03-09 Thread Jan Urbański
Hi,

I just wanted to thank everyone involved in shepherding the PL/Python
patches into the master repo, the testers, reviewers, commenters and
especially Peter, for their help and diligence.

The outstanding tracebacks patch is still being worked on, but
irrelevant of whether it will make it or not, I think PL/Python is now
in a significantly better shape than when we started.

Thanks again!
Jan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-09 Thread Magnus Hagander
On Wed, Mar 9, 2011 at 08:38, Fujii Masao masao.fu...@gmail.com wrote:
 On Wed, Mar 9, 2011 at 2:14 PM, Jaime Casanova ja...@2ndquadrant.com wrote:
 On Tue, Mar 8, 2011 at 11:58 AM, Robert Haas robertmh...@gmail.com wrote:

 The fast shutdown handling seems fine, but why not just handle smart
 shutdown the same way?

 currently, smart shutdown means no new connections, wait until
 existing ones close normally. for consistency, it should behave the
 same for sync rep.

 Agreed. I think that user who wants to request smart shutdown expects all
 the existing connections to basically be closed normally by the client. So it
 doesn't seem to be good idea to forcibly close the connection and prevent
 the COMMIT from being returned in smart shutdown case. But I'm all ears
 for better suggestions.

don't use smart shutdowns? ;)

Anyway, for those that *do* use smart intentionally, I agree that
doing any kind of forced close at all is just plain wrong.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Beginner question: Hacking environment?

2011-03-09 Thread Adrian von Bidder
[adding webmaster to cc]

On Tuesday 08 March 2011 21.20:20 Andres Freund wrote:
  create account, ...
 Its linked on the mainpage: http://www.postgresql.org/community/signup

Hmm.  Could it be that this web form doesn't have a mail queue and thus 
doesn't retry to send the mail when the first attempt was denied by 
greylisting?  And now it says user already exists...

(At least: I hven't seen a retry for 4h, first mail was rejected at 08:42 
+0100)

cheers
-- vbi

-- 
featured product: Debian GNU/Linux - http://debian.org


signature.asc
Description: This is a digitally signed message part.


Re: [webmaster] [HACKERS] Beginner question: Hacking environment?

2011-03-09 Thread Magnus Hagander
On Wed, Mar 9, 2011 at 13:00, Adrian von Bidder avbid...@fortytwo.ch wrote:
 [adding webmaster to cc]

 On Tuesday 08 March 2011 21.20:20 Andres Freund wrote:
  create account, ...
 Its linked on the mainpage: http://www.postgresql.org/community/signup

 Hmm.  Could it be that this web form doesn't have a mail queue and thus
 doesn't retry to send the mail when the first attempt was denied by
 greylisting?  And now it says user already exists...

It definitely does have this.


I see a graylisted email that's in the queue... I'll give it a kick,
but normally you jsut have to wait...

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [webmaster] [HACKERS] Beginner question: Hacking environment?

2011-03-09 Thread Adrian von Bidder
On Wednesday 09 March 2011 13.05:55 Magnus Hagander wrote:
 I see a graylisted email that's in the queue... I'll give it a kick,
 but normally you jsut have to wait...

Thanks, it arrived.

I'm used to wait when I enable greylisting. 4h delay is rare, though.

greets
-- vbi

-- 
I liken ISPs to sausage factories.  You know what goes in, and you think
that what comes out is tasty, but by Apollo's unwashed speedos you DON'T
want to know what goes on in between.
-- Sid Dabster


signature.asc
Description: This is a digitally signed message part.


Re: [webmaster] [HACKERS] Beginner question: Hacking environment?

2011-03-09 Thread Alvaro Herrera
Excerpts from Adrian von Bidder's message of mié mar 09 09:13:04 -0300 2011:
 On Wednesday 09 March 2011 13.05:55 Magnus Hagander wrote:
  I see a graylisted email that's in the queue... I'll give it a kick,
  but normally you jsut have to wait...
 
 Thanks, it arrived.
 
 I'm used to wait when I enable greylisting. 4h delay is rare, though.

Yeah, I think these servers wait for an excessive period if the initial
delivery fails.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Fwd: index corruption in PG 8.3.13

2011-03-09 Thread Nikhil Sontakke
Re-sending without the attachments. Can someone please allow my
attachments through from the previous email?

TIA
Nikhils

-- Forwarded message --
From: Nikhil Sontakke nikhil.sonta...@enterprisedb.com
Date: Wed, Mar 9, 2011 at 8:42 PM
Subject: index corruption in PG 8.3.13
To: pgsql-hackers@postgresql.org


Hi,

I am currently looking at a physical file belonging to a corrupt
index. This is with PG 8.3.13 on an ext3 filesystem on suse linux with
a RAID1 setup. The sequence of operation seems nothing suspicious.

Create table, create a bunch of indexes on that table and do
insert/update activity. One difference here is that VACUUM FULL might
be run in between along with the usual autovacuum mechanism when the
inserts/updates are going on.

So one of the vacuum calls errored out with a corrupt (completely
zeroed out in this case) block with the usual hint to reindex.
This is a single machine setup with no slaves. Some analysis using
pg_filedump comes up with the following:

There are a total of 525 blocks (numbered 0 to 524). The root page is
block #3. The corrupt or zeroed out page is block #523.

The basic issue is that item2 in root block points to block 523 which
is completely zeroed out and this has broken the index chain
completely!

Other peculiarity in the index file is that we found a lot of zeroed
out pages. Blocks from #279 to #518 are all completely zeroed out
without any signs of even a page header. Any ideas on how we can get
so many zeroed out blocks? Apart from the extend code path, I fail to
see any other. And this is an unusually large number of zero pages...

Blocks 519 to 521 are DELETED. They do not have the LEAF flag set,
meaning they could be internal pages, but that is strange since ROOT
page is at level 1. Also importantly their next XID is set FrozenXid,
meaning VACUUM FULL was at play. Maybe due to deletes, we reduced the
hierarchy level or something?

I am attaching the data file, the binary and the 8.3 compliant
pg_filedump sources with this mail. Just copy the pg_filedump
directory into contrib and use Makefile.contrib to compile. Cannot
attach the output of running pg_filedump since my email will cross the
attachment limits. Am also attaching my dotty attempts via a png file
to visually show the scenario, hopefully that should make things
clearer.

Feedback, comments, insights appreciated. Appears to be a tough nut to
crack :( :), and although blaming the hardware for the zero pages
might come to mind, still the deletion activity done by Vacuum/Vacuum
Full seems a bit suspicious.

Regards,
Nikhils

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: index corruption in PG 8.3.13

2011-03-09 Thread Alvaro Herrera
Excerpts from Nikhil Sontakke's message of mié mar 09 10:51:50 -0300 2011:
 Re-sending without the attachments. Can someone please allow my
 attachments through from the previous email?

They are not in the moderation queue, so presumably they got eaten by
the antispam grue.


 Blocks 519 to 521 are DELETED. They do not have the LEAF flag set,
 meaning they could be internal pages, but that is strange since ROOT
 page is at level 1. Also importantly their next XID is set FrozenXid,
 meaning VACUUM FULL was at play. Maybe due to deletes, we reduced the
 hierarchy level or something?

Hierarchy level is never reduced.

I'll send you a perl program we wrote for a customer to check for
strange issues in btrees.  Please give it a spin; it may give you more
clues.  If you find additional checks to add, please let me know!

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-09 Thread Simon Riggs
On Wed, 2011-03-09 at 16:38 +0900, Fujii Masao wrote:
 On Wed, Mar 9, 2011 at 2:14 PM, Jaime Casanova ja...@2ndquadrant.com wrote:
  On Tue, Mar 8, 2011 at 11:58 AM, Robert Haas robertmh...@gmail.com wrote:
 
  The fast shutdown handling seems fine, but why not just handle smart
  shutdown the same way?
 
  currently, smart shutdown means no new connections, wait until
  existing ones close normally. for consistency, it should behave the
  same for sync rep.
 
 Agreed. I think that user who wants to request smart shutdown expects all
 the existing connections to basically be closed normally by the client. So it
 doesn't seem to be good idea to forcibly close the connection and prevent
 the COMMIT from being returned in smart shutdown case. But I'm all ears
 for better suggestions.
 
 Anyway, we got the consensus about how fast shutdown should work with
 sync rep. So I created the patch. Please feel free to comment and commit
 the patch first ;)

We're just about to publish Alpha4 with this feature in.

If we release waiters too early we will cause effective data loss, that
part is agreed. We've also accepted that there are few ways to release
the waiters.

I want to release the first version as safe and then relax from there
after feedback. What I don't want to hear is lots of complaints or
arguments about data loss from the first version. We can relax later,
after some time and thought.

So for now, I don't want to apply this patch or other similar ones that
seek to release waiters in various circumstances. That isn't a rejection
of you, its just a wish to play it safe and slowly.

-- 
 Simon Riggs   http://www.2ndQuadrant.com/books/
 PostgreSQL Development, 24x7 Support, Training and Services
 


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: index corruption in PG 8.3.13

2011-03-09 Thread Alvaro Herrera
Excerpts from Nikhil Sontakke's message of mié mar 09 11:16:22 -0300 2011:
  Re-sending without the attachments. Can someone please allow my
  attachments through from the previous email?
 
  They are not in the moderation queue, so presumably they got eaten by
  the antispam grue.
 
 Ouch. Attempting to attach the dotty image again..

I don't understand this graph.  What are the arrows?  Downlinks or
sibling pointers?

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-09 Thread Yeb Havinga

On 2011-03-09 15:10, Simon Riggs wrote:

On Wed, 2011-03-09 at 16:38 +0900, Fujii Masao wrote:

On Wed, Mar 9, 2011 at 2:14 PM, Jaime Casanovaja...@2ndquadrant.com  wrote:

On Tue, Mar 8, 2011 at 11:58 AM, Robert Haasrobertmh...@gmail.com  wrote:

The fast shutdown handling seems fine, but why not just handle smart
shutdown the same way?

currently, smart shutdown means no new connections, wait until
existing ones close normally. for consistency, it should behave the
same for sync rep.

Agreed. I think that user who wants to request smart shutdown expects all
the existing connections to basically be closed normally by the client. So it
doesn't seem to be good idea to forcibly close the connection and prevent
the COMMIT from being returned in smart shutdown case. But I'm all ears
for better suggestions.

Anyway, we got the consensus about how fast shutdown should work with
sync rep. So I created the patch. Please feel free to comment and commit
the patch first ;)

We're just about to publish Alpha4 with this feature in.

If we release waiters too early we will cause effective data loss, that
part is agreed. We've also accepted that there are few ways to release
the waiters.

I want to release the first version as safe and then relax from there
after feedback.

This is not safe and possible in the first version:

1) issue stop on master when no sync standby is connected:
mgrid@mg73:~$ pg_ctl -D /data stop
waiting for server to shut 
down... failed

pg_ctl: server does not shut down

2) start the standby that failed
mgrid@mg72:~$ pg_ctl -D /data start
pg_ctl: another server might be running; trying to start server anyway
LOG:  0: database system was interrupted while in recovery at log 
time 2011-03-09 15:22:31 CET
HINT:  If this has occurred more than once some data might be corrupted 
and you might need to choose an earlier recovery target.

LOG:  0: entering standby mode
LOG:  0: redo starts at 57/1A78
LOG:  0: consistent recovery state reached at 57/1AA0
FATAL:  XX000: could not connect to the primary server: FATAL:  the 
database system is shutting down


LOCATION:  libpqrcv_connect, libpqwalreceiver.c:102
server starting
mgrid@mg72:~$ FATAL:  XX000: could not connect to the primary server: 
FATAL:  the database system is shutting down


A safe solution would be to prevent smart shutdown on the master if it 
is in sync mode and there are no sync standbys connected.


The current situation is definately unsafe because it forces people that 
are in this state to do a fast shutdown.. but that fails as well, so 
they are only left with immediate.


mgrid@mg73:~$ pg_ctl -D /data stop -m fast
waiting for server to shut 
down... failed

pg_ctl: server does not shut down
mgrid@mg73:~$ pg_ctl -D /data stop -m immediate
waiting for server to shut down done
server stopped

regards,
Yeb Havinga


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: index corruption in PG 8.3.13

2011-03-09 Thread Nikhil Sontakke
 Ouch. Attempting to attach the dotty image again..

 I don't understand this graph.  What are the arrows?  Downlinks or
 sibling pointers?


Sorry, they are sibling previous and next pointers. The ROOT is at
level 1, rest all live blocks are at level 0. #524 is the leftmost
page.

Regards,
Nikhils

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-09 Thread Simon Riggs
On Wed, 2011-03-09 at 15:37 +0100, Yeb Havinga wrote:

 The current situation is definately unsafe because it forces people
 that are in this state to do a fast shutdown.. but that fails as well,
 so they are only left with immediate.

All the more reason not to change anything, since we disagree.

The idea is that you're supposed to wait for the standby to come back up
or do failover. If you shutdown the master its because you are choosing
to failover.

Shutting down the master and restarting without failover leads to a
situation where some sync rep commits are not on both master and
standby. Making it easier to shutdown encourages that, which I don't
wish to do, at this stage.

We may decide that this is the right approach but I don't wish to rush
into that decision. I want to have clear agreement about all the changes
we want and what we call them if we do them. Zero data loss is
ultimately about users having confidence in us, not about specific
features. Our disagreements on this patch risk damaging that confidence,
whoever is right/wrong.

Further changes can be made over the course of the next few weeks, based
upon feedback from a wider pool of potential users.

-- 
 Simon Riggs   http://www.2ndQuadrant.com/books/
 PostgreSQL Development, 24x7 Support, Training and Services
 


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] wrap alpha4 tomorrow ~9am Eastern (was: Alpha4 release blockers)

2011-03-09 Thread Robert Haas
On Tue, Mar 8, 2011 at 12:00 PM, Robert Haas robertmh...@gmail.com wrote:
 Going once, going twice...

 I'll go ahead and do this, barring objections or some other volunteer.

developer.postgresql.org apparently hates me.  After waiting an
insanely long time to copy over the exported tarball to that machine,
I tried to follow the instructions at
http://wiki.postgresql.org/wiki/Alpha_release_process but of course it
didn't work:

$ src/tools/version_stamp.pl alpha4
Stamped these files with version number 9.1alpha4:
configure.in
doc/bug.template
src/include/pg_config.h.win32
src/interfaces/libpq/libpq.rc.in
src/port/win32ver.rc
Don't forget to run autoconf 2.63 before committing.
$ autoconf --version
autoconf (GNU Autoconf) 2.62
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv2+: GNU GPL version 2 or later
http://gnu.org/licenses/old-licenses/gpl-2.0.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David J. MacKenzie and Akim Demaille.
$ autoconf
configure.in:22: error: Autoconf version 2.63 is required.
Untested combinations of 'autoconf' and PostgreSQL versions are not
recommended.  You can remove the check from 'configure.in' but it is then
your responsibility whether the result works or not.
configure.in:22: the top level
autom4te-2.62: /usr/local/bin/gm4 failed with exit status: 1

As Magnus pointed out to me on IM, there must be a usable version of
autoconf on this machine somewhere if it's doing the nightly snapshot
builds, but beats me where it is.  Help?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] wrap alpha4 tomorrow ~9am Eastern (was: Alpha4 release blockers)

2011-03-09 Thread Marc G. Fournier


Due to backbranch packaging, and having to support several different 
versions of autoconf as a results, its a bit more confusing ...


'k, normally it would be in /usr/local/bin:

developer# ls -lt autoconf-*
-r-xr-xr-x  1 root  wheel  14657 Aug 13  2009 autoconf-2.62
-r-xr-xr-x  1 root  wheel  13420 Aug 26  2007 autoconf-2.61
-r-xr-xr-x  1 root  wheel   5009 Aug 16  2007 autoconf-2.13
-r-xr-xr-x  1 root  wheel   7674 Aug 16  2007 autoconf-2.59
-r-xr-xr-x  1 root  wheel   6196 Aug 16  2007 autoconf-2.53

But, since they seem to have 'skipped' 2.63 in FreeBSD for some reason 
(current version in ports is 2.68), 2.63 had to be manually installed 
seperately from ports:


developer# /root/bin/autoconf --version
autoconf (GNU Autoconf) 2.63
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv2+: GNU GPL version 2 or later
http://gnu.org/licenses/old-licenses/gpl-2.0.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David J. MacKenzie and Akim Demaille.

Sorry for the confusion on that ...


On Wed, 9 Mar 2011, Robert Haas wrote:


On Tue, Mar 8, 2011 at 12:00 PM, Robert Haas robertmh...@gmail.com wrote:

Going once, going twice...

I'll go ahead and do this, barring objections or some other volunteer.


developer.postgresql.org apparently hates me.  After waiting an
insanely long time to copy over the exported tarball to that machine,
I tried to follow the instructions at
http://wiki.postgresql.org/wiki/Alpha_release_process but of course it
didn't work:

$ src/tools/version_stamp.pl alpha4
Stamped these files with version number 9.1alpha4:
configure.in
doc/bug.template
src/include/pg_config.h.win32
src/interfaces/libpq/libpq.rc.in
src/port/win32ver.rc
Don't forget to run autoconf 2.63 before committing.
$ autoconf --version
autoconf (GNU Autoconf) 2.62
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv2+: GNU GPL version 2 or later
http://gnu.org/licenses/old-licenses/gpl-2.0.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David J. MacKenzie and Akim Demaille.
$ autoconf
configure.in:22: error: Autoconf version 2.63 is required.
Untested combinations of 'autoconf' and PostgreSQL versions are not
recommended.  You can remove the check from 'configure.in' but it is then
your responsibility whether the result works or not.
configure.in:22: the top level
autom4te-2.62: /usr/local/bin/gm4 failed with exit status: 1

As Magnus pointed out to me on IM, there must be a usable version of
autoconf on this machine somewhere if it's doing the nightly snapshot
builds, but beats me where it is.  Help?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company




Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] wrap alpha4 tomorrow ~9am Eastern (was: Alpha4 release blockers)

2011-03-09 Thread Peter Eisentraut
On ons, 2011-03-09 at 10:12 -0500, Robert Haas wrote:
 developer.postgresql.org apparently hates me.  After waiting an
 insanely long time to copy over the exported tarball to that machine,

I run the export on d.p.o.

 As Magnus pointed out to me on IM, there must be a usable version of
 autoconf on this machine somewhere if it's doing the nightly snapshot
 builds, but beats me where it is.  Help?

The snapshot doesn't need autoconf, but there is an installation in
~petere/sw/ac263/ that you can use.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] wrap alpha4 tomorrow ~9am Eastern (was: Alpha4 release blockers)

2011-03-09 Thread Robert Haas
On Wed, Mar 9, 2011 at 10:35 AM, Peter Eisentraut pete...@gmx.net wrote:
 On ons, 2011-03-09 at 10:12 -0500, Robert Haas wrote:
 developer.postgresql.org apparently hates me.  After waiting an
 insanely long time to copy over the exported tarball to that machine,

 I run the export on d.p.o.

I can see why, but I didn't have a git clone on that machine.

 As Magnus pointed out to me on IM, there must be a usable version of
 autoconf on this machine somewhere if it's doing the nightly snapshot
 builds, but beats me where it is.  Help?

 The snapshot doesn't need autoconf, but there is an installation in
 ~petere/sw/ac263/ that you can use.

OK, thanks.  Running make distcheck now.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] wrap alpha4 tomorrow ~9am Eastern (was: Alpha4 release blockers)

2011-03-09 Thread Robert Haas
On Wed, Mar 9, 2011 at 10:43 AM, Robert Haas robertmh...@gmail.com wrote:
 On Wed, Mar 9, 2011 at 10:35 AM, Peter Eisentraut pete...@gmx.net wrote:
 On ons, 2011-03-09 at 10:12 -0500, Robert Haas wrote:
 developer.postgresql.org apparently hates me.  After waiting an
 insanely long time to copy over the exported tarball to that machine,

 I run the export on d.p.o.

 I can see why, but I didn't have a git clone on that machine.

 As Magnus pointed out to me on IM, there must be a usable version of
 autoconf on this machine somewhere if it's doing the nightly snapshot
 builds, but beats me where it is.  Help?

 The snapshot doesn't need autoconf, but there is an installation in
 ~petere/sw/ac263/ that you can use.

 OK, thanks.  Running make distcheck now.

And done.

Files now up at:

http://developer.postgresql.org/~rhaas/

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Update of replication/README

2011-03-09 Thread Robert Haas
On Wed, Mar 9, 2011 at 6:11 AM, Fujii Masao masao.fu...@gmail.com wrote:
 The attached patch updates replication/README to reflect current
 walsender/walreceiver behavior. It doesn't include any description
 about sync rep. That would need to be added later.

Hrm.  What about this hunk?

-Each walsender allocates an entry from the WalSndCtl array, and advertises
-there how far it has streamed WAL already. This is used at checkpoints, to
-avoid recycling WAL that hasn't been streamed to a slave yet. However,
-that doesn't stop such WAL from being recycled when the connection is not
-established.
+Each walsender allocates an entry from the WalSndCtl array, and tracks
+information about replication progress. User can monitor them via
+statistics views.

Is the deleted text not (or no longer) true?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] How should the waiting backends behave in sync rep?

2011-03-09 Thread Robert Haas
On Tue, Mar 8, 2011 at 10:06 AM, Fujii Masao masao.fu...@gmail.com wrote:
 How should the backends waiting for replication behave when
 synchrnous_standby_names
 is set to '' and the configuration file is reloaded? Now they keep
 waiting for the ACK from the
 standby. But I think that it's more natural for them to get out of the
 wait state and complete
 the transaction in that case. If we'll change them in that way, we
 would no longer need
 something like pg_ctl standalone which I mentioned in another thread. 
 Thought?

I think so.  Looking at assign_synchronous_standby_names, the
following code just looks wrong:

if (doit  list_length(elemlist)  0)
sync_standbys_defined = true;

Once sync_standbys_defined becomes true, there's no way for it to ever
become false again.  That can't be right.  That means that if you
disable sync rep by zeroing out synchronous_standby_names, backends
that already existed at the time you made the change will continue to
act as though sync rep is enabled until they exit.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] wrap alpha4 tomorrow ~9am Eastern (was: Alpha4 release blockers)

2011-03-09 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote:
 
 Files now up at:
 
 http://developer.postgresql.org/~rhaas/
 
As an initial sanity test I downloaded the bz2 version and its md5
file.  The md5sum checked out and this all ran as expected:
 
tar -xjf postgresql-9.1alpha4.tar.bz2
cd postgresql-9.1alpha4/
./configure --prefix=/usr/local/pgsql-9.1alpha4 \
--enable-debug \
--enable-cassert \
--enable-depend
make world
make check-world
pushd src/test/isolation/ ; make check ; popd
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] wrap alpha4 tomorrow ~9am Eastern (was: Alpha4 release blockers)

2011-03-09 Thread Robert Haas
On Wed, Mar 9, 2011 at 12:30 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
 Robert Haas robertmh...@gmail.com wrote:

 Files now up at:

 http://developer.postgresql.org/~rhaas/

 As an initial sanity test I downloaded the bz2 version and its md5
 file.  The md5sum checked out and this all ran as expected:

 tar -xjf postgresql-9.1alpha4.tar.bz2
 cd postgresql-9.1alpha4/
 ./configure --prefix=/usr/local/pgsql-9.1alpha4 \
            --enable-debug \
            --enable-cassert \
            --enable-depend
 make world
 make check-world
 pushd src/test/isolation/ ; make check ; popd

Thanks.  I copied the files to ~ftp/pub/source/9.1alpha4 on
developer.postgresql.org also, along side the existing 9.1alpha{1,2,3}
directories, but I'm fuzzy on how it all gets pushed out to wwwmaster
and the mirror network.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] wrap alpha4 tomorrow ~9am Eastern (was: Alpha4 release blockers)

2011-03-09 Thread Magnus Hagander
On Wed, Mar 9, 2011 at 18:53, Robert Haas robertmh...@gmail.com wrote:
 On Wed, Mar 9, 2011 at 12:30 PM, Kevin Grittner
 kevin.gritt...@wicourts.gov wrote:
 Robert Haas robertmh...@gmail.com wrote:

 Files now up at:

 http://developer.postgresql.org/~rhaas/

 As an initial sanity test I downloaded the bz2 version and its md5
 file.  The md5sum checked out and this all ran as expected:

 tar -xjf postgresql-9.1alpha4.tar.bz2
 cd postgresql-9.1alpha4/
 ./configure --prefix=/usr/local/pgsql-9.1alpha4 \
            --enable-debug \
            --enable-cassert \
            --enable-depend
 make world
 make check-world
 pushd src/test/isolation/ ; make check ; popd

 Thanks.  I copied the files to ~ftp/pub/source/9.1alpha4 on
 developer.postgresql.org also, along side the existing 9.1alpha{1,2,3}
 directories, but I'm fuzzy on how it all gets pushed out to wwwmaster
 and the mirror network.

Time will take care of that for you. You just have a coffee and wait...

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Native XML

2011-03-09 Thread Bruce Momjian
Robert Haas wrote:
 On Mon, Feb 28, 2011 at 10:30 AM, Tom Lane t...@sss.pgh.pa.us wrote:
  Well, in principle we could allow them to work on both, just the same
  way that (for instance) + is a standardized operator but works on more
  than one datatype. ?But I agree that the prospect of two parallel types
  with essentially duplicate functionality isn't pleasing at all.
 
 The real issue here is whether we want to store XML as text (as we do
 now) or as some predigested form which would make output the whole
 thing slower but speed up things like xpath lookups.  We had the same
 issue with JSON, and due to the uncertainty about which way to go with
 it we ended up integrating nothing into core at all.  It's really not
 clear that there is one way of doing this that is right for all use
 cases.  If you are storing xml in an xml column just to get it
 validated, and doing no processing in the DB, then you'd probably
 prefer our current representation.  If you want to build functional
 indexes on xpath expressions, and then run queries that extract data
 using other xpath expressions, you would probably prefer the other
 representation.

Someone should measure how much overhead the indexing of xml values
might have.  If it is minor, we might be OK with only an indexed xml
type.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Native XML

2011-03-09 Thread Robert Haas
On Wed, Mar 9, 2011 at 1:11 PM, Bruce Momjian br...@momjian.us wrote:
 Robert Haas wrote:
 On Mon, Feb 28, 2011 at 10:30 AM, Tom Lane t...@sss.pgh.pa.us wrote:
  Well, in principle we could allow them to work on both, just the same
  way that (for instance) + is a standardized operator but works on more
  than one datatype. ?But I agree that the prospect of two parallel types
  with essentially duplicate functionality isn't pleasing at all.

 The real issue here is whether we want to store XML as text (as we do
 now) or as some predigested form which would make output the whole
 thing slower but speed up things like xpath lookups.  We had the same
 issue with JSON, and due to the uncertainty about which way to go with
 it we ended up integrating nothing into core at all.  It's really not
 clear that there is one way of doing this that is right for all use
 cases.  If you are storing xml in an xml column just to get it
 validated, and doing no processing in the DB, then you'd probably
 prefer our current representation.  If you want to build functional
 indexes on xpath expressions, and then run queries that extract data
 using other xpath expressions, you would probably prefer the other
 representation.

 Someone should measure how much overhead the indexing of xml values
 might have.  If it is minor, we might be OK with only an indexed xml
 type.

I think the relevant thing to measure would be how fast the
predigested representation speeds up the evaluation of xpath
expressions.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] wrap alpha4 tomorrow ~9am Eastern (was: Alpha4 release blockers)

2011-03-09 Thread Robert Haas
On Wed, Mar 9, 2011 at 12:56 PM, Magnus Hagander mag...@hagander.net wrote:
 Time will take care of that for you. You just have a coffee and wait...

Time seems to  have done the trick (though my coffee would be getting
cold by now).  I now see it at:

http://www.postgresql.org/ftp/source/v9.1alpha4/

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Native XML

2011-03-09 Thread Yeb Havinga

On 2011-03-09 19:30, Robert Haas wrote:

On Wed, Mar 9, 2011 at 1:11 PM, Bruce Momjianbr...@momjian.us  wrote:

Robert Haas wrote:

On Mon, Feb 28, 2011 at 10:30 AM, Tom Lanet...@sss.pgh.pa.us  wrote:

Well, in principle we could allow them to work on both, just the same
way that (for instance) + is a standardized operator but works on more
than one datatype. ?But I agree that the prospect of two parallel types
with essentially duplicate functionality isn't pleasing at all.

The real issue here is whether we want to store XML as text (as we do
now) or as some predigested form which would make output the whole
thing slower but speed up things like xpath lookups.  We had the same
issue with JSON, and due to the uncertainty about which way to go with
it we ended up integrating nothing into core at all.  It's really not
clear that there is one way of doing this that is right for all use
cases.  If you are storing xml in an xml column just to get it
validated, and doing no processing in the DB, then you'd probably
prefer our current representation.  If you want to build functional
indexes on xpath expressions, and then run queries that extract data
using other xpath expressions, you would probably prefer the other
representation.

Someone should measure how much overhead the indexing of xml values
might have.  If it is minor, we might be OK with only an indexed xml
type.

I think the relevant thing to measure would be how fast the
predigested representation speeds up the evaluation of xpath
expressions.
About a predigested representation, I hope I'm not insulting anyone's 
education here, but a lot of XML database 'accellerators' seem to be 
using the pre and post orders (see 
http://en.wikipedia.org/wiki/Tree_traversal) of the document nodes. The 
following two pdfs show how these orders can be used to query for e.g. 
all ancestors of a node: second pdf slide 10: for nodes x,y : x is an 
ancestor of y when x.pre  y.pre AND x.post  y.post.


www.cse.unsw.edu.au/~cs4317/09s1/tutorials/tutor4.pdf  about the format
www.cse.unsw.edu.au/~cs4317/09s1/tutorials/tutor10.pdf about querying 
the format


regards,
Yeb Havinga



Re: [HACKERS] Problem with pg_upgrade (8.4 - 9.0) due to ALTER DATABASE SET ROLE

2011-03-09 Thread Bruce Momjian
Robert Treat wrote:
 On Mon, Feb 28, 2011 at 3:42 AM, Magnus Hagander mag...@hagander.net wrote:
  On Mon, Feb 28, 2011 at 06:21, Tom Lane t...@sss.pgh.pa.us wrote:
  Robert Treat r...@xzilla.net writes:
  Did anything ever come of this discussion?
 
  I think it's a TODO --- nothing done about it as yet, AFAIR.
 
  On one of the databases I
  was upgrading, I ran into a similar problem with roles that are set as
  roles. The problem seems to stem from pg_dumpall dumping roles in
  alphabetical order:
 
  CREATE ROLE asha;
  ALTER ROLE asha SET role TO 'omniti';
  .. sometime later ...
  CREATE ROLE omniti;
 
  That seems like a pretty bizarre thing to do. ?Why would you want such a
  setting?
 
  I'm sure there are several. I've seen (and done) this more than once
  to ensure that the owner of newly created object is the shared role
  and not the individual, for example.
 
 
 Yeah, there are actually several of the roles that get set to the
 omniti role, like the robert role, which doesn't have any issue
 because it comes alphabetically after omniti. This also helps folks
 get around several permission related issues (simplified management,
 uniform permissions across users, simplified dependencies, etc..), but
 object ownership is a key part of it.

Added to TODO:

Allow pg_dumpall to output restorable ALTER USER/DATABASE SET settings

* http://archives.postgresql.org/pgsql-hackers/2010-12/msg00916.php
* http://archives.postgresql.org/pgsql-hackers/2011-01/msg00394.php
* http://archives.postgresql.org/pgsql-hackers/2011-02/msg02359.php 

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: psql include file using relative path

2011-03-09 Thread David Fetter
Being able to include relative paths is a really great feature, but
should it have a UI (well, API) distinct from fixed-path includes?  My
first instinct is that it shouldn't, but I haven't really thought it
through thoroughly.

Cheers,
David (the tough coughs as he ploughs the dough)
On Tue, Mar 08, 2011 at 03:19:40PM -0500, Gurjeet Singh wrote:
 Attached patch implements tab completion. It also introduces the long-form
 alternative \include_relative for \ir
 
 Regards,
 
 On Tue, Mar 8, 2011 at 2:18 PM, Ibrar Ahmed ibrar.ah...@gmail.com wrote:
 
  Gurjeet!
 
  What about tab completion, like in \i command?
 
  On Fri, Feb 25, 2011 at 5:07 AM, Robert Haas robertmh...@gmail.com
  wrote:
   On Thu, Feb 24, 2011 at 6:21 PM, Gurjeet Singh singh.gurj...@gmail.com
  wrote:
   psql has the ability to execute commands from a file, but if one
  wishes
   to develop and provide a modularized set of sql files, then psql is not
  very
   helpful because the \i command can open file paths either if they are
   absolute paths or if they are palced correctly relative to psql's
  current
   working directory.
  
   Attached patch adds a new meta-command to psql, '\ir' that allows the
  user
   to process files relative to currently processing file.
  
   Please add this patch to the currently open CommitFest at:
  
   https://commitfest.postgresql.org/action/commitfest_view/open
  
   --
   Robert Haas
   EnterpriseDB: http://www.enterprisedb.com
   The Enterprise PostgreSQL Company
  
   --
   Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
   To make changes to your subscription:
   http://www.postgresql.org/mailpref/pgsql-hackers
  
 
 
 
  --
 Ibrar Ahmed
 
 
 
 
 -- 
 Gurjeet Singh
 EnterpriseDB http://www.enterprisedb.com/ Corporation
 The Enterprise PostgreSQL http://www.postgresql.org/ Company


 
 -- 
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers


-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] FuncExpr.collid/OpExpr.collid unworkably serving double duty

2011-03-09 Thread Tom Lane
So I was moving some error checks around and all of a sudden the
regression tests blew up on me, with lots of errors about how type X
didn't support collations (which indeed it didn't).  After some
investigation I realized what should have been apparent much earlier:
the collations patch is trying to use one field for two different
purposes.  In particular, collid in FuncExpr and related nodes is
used in both of these ways:

* as the collation to apply during execution of the function;

* as the collation of the function's result.

The trouble is that these usages are only compatible if both the
arguments and result of the function are of collatable types.  In
particular, the change I made was causing CREATE VIEW to fail on
examples like this:

regression=# create view vv as select 'z'::text  'y'::text as b;
ERROR:  collations are not supported by type boolean

because the OpExpr node must have nonzero collid to do a textual
comparison, but then exprCollation() claims that the result of the
expression has that collation too, which it does not because it's
only bool.

Aside from confusing code that tries to impute a collation to the
result of an expression, this will confuse the collation assignment code
itself: select_common_collation will mistakenly believe that
non-collatable input arguments should affect its results.  I'm not sure
how you managed to avoid such failures in the committed patch (if indeed
you did avoid them and they're not just lurking in un-regression-tested
cases); but in any case it seems far too fragile to keep it this way.

There are basically two things we could do about this:

1. Add two fields not one to nodes representing function/operator calls.

2. Change exprCollation() to do a type_is_collatable check on the
node result type before believing that the collid field is relevant.

Of course the syscache lookup implied by type_is_collatable will mean
that exprCollation is orders of magnitude slower than it is now.  So
this is a pretty straightforward space vs speed tradeoff.  I'm inclined
to think that choice #1 is the lesser evil, because I'm afraid that this
patch has already added an undesirable number of new cache lookups to
the basic expression parsing paths.

Thoughts?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Native XML

2011-03-09 Thread Anton
On 03/09/2011 08:21 PM, Yeb Havinga wrote:
 On 2011-03-09 19:30, Robert Haas wrote:
 On Wed, Mar 9, 2011 at 1:11 PM, Bruce Momjian br...@momjian.us wrote:
 
 Robert Haas wrote:
   
 On Mon, Feb 28, 2011 at 10:30 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 
 Well, in principle we could allow them to work on both, just the same
 way that (for instance) + is a standardized operator but works on more
 than one datatype. ?But I agree that the prospect of two parallel types
 with essentially duplicate functionality isn't pleasing at all.
   
 The real issue here is whether we want to store XML as text (as we do
 now) or as some predigested form which would make output the whole
 thing slower but speed up things like xpath lookups.  We had the same
 issue with JSON, and due to the uncertainty about which way to go with
 it we ended up integrating nothing into core at all.  It's really not
 clear that there is one way of doing this that is right for all use
 cases.  If you are storing xml in an xml column just to get it
 validated, and doing no processing in the DB, then you'd probably
 prefer our current representation.  If you want to build functional
 indexes on xpath expressions, and then run queries that extract data
 using other xpath expressions, you would probably prefer the other
 representation.
 
 Someone should measure how much overhead the indexing of xml values
 might have.  If it is minor, we might be OK with only an indexed xml
 type.
   
 I think the relevant thing to measure would be how fast the
 predigested representation speeds up the evaluation of xpath
 expressions.
 
 About a predigested representation, I hope I'm not insulting anyone's
 education here, but a lot of XML database 'accellerators' seem to be
 using the pre and post orders (see
 http://en.wikipedia.org/wiki/Tree_traversal) of the document nodes.
 The following two pdfs show how these orders can be used to query for
 e.g. all ancestors of a node: second pdf slide 10: for nodes x,y : x
 is an ancestor of y when x.pre  y.pre AND x.post  y.post.

 www.cse.unsw.edu.au/~cs4317/09s1/tutorials/tutor4.pdf  about the format
 www.cse.unsw.edu.au/~cs4317/09s1/tutorials/tutor10.pdf about querying
 the format

 regards,
 Yeb Havinga

This looks rather like a special kind of XML shredding and that is
listed at http://wiki.postgresql.org/wiki/Todo

About the predigested / plain storage and the evaluation: I haven't yet
fully given up the idea to play with it, even though on purely
experimental basis (i.e. with little or no ambition to contribute to the
core product). If doing so, interesting might also be to use TOAST
slicing during the xpath evaluation so that large documents are not
fetched immediately as a whole, if the xpath is rather 'short'.

But first I should let all the thoughts 'settle down'. There may well be
other areas of Postgres where it's worth to spend some time, whether
writing something or just reading.


[HACKERS] select_common_collation callers way too ready to throw error

2011-03-09 Thread Tom Lane
db1=# create table t1 (f1 text collate aa_DJ.utf8,
  f2 text collate an_ES.utf8 );
CREATE TABLE
db1=# select f1  f2 from t1;
ERROR:  collation mismatch between implicit collations aa_DJ.utf8 and 
an_ES.utf8
LINE 1: select f1  f2 from t1;
^
HINT:  You can override the collation by applying the COLLATE clause to one or 
both expressions.
db1=# select f1 || f2 from t1;
ERROR:  collation mismatch between implicit collations aa_DJ.utf8 and 
an_ES.utf8
LINE 1: select f1 || f2 from t1;
 ^
HINT:  You can override the collation by applying the COLLATE clause to one or 
both expressions.

The first of these errors is OK, but surely the second is not, because ||
doesn't give a fig about collations.  I think instead of this:

/* XXX: If we knew which functions required collation information,
 * we could selectively set the last argument to true here. */
funccollid = select_common_collation(pstate, fargs, false);

we need:

/*
 * If we knew which functions required collation information,
 * we could selectively set the last argument to false here,
 * allowing the error to be reported at parse time not runtime.
 */
funccollid = select_common_collation(pstate, fargs, true);

Now the downside of that is that a runtime failure won't give you an
parse error pointer to indicate which function is having trouble ...
but having an error pointer for an error that shouldn't be thrown in
the first place is small consolation.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-03-09 Thread Mark Kirkwood

New version:

- adds documentation
- adds category RESOURCES_DISK


temp-files-v2.patch.gz
Description: GNU Zip compressed data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: index corruption in PG 8.3.13

2011-03-09 Thread Nikhil Sontakke
Hi,

 Blocks 519 to 521 are DELETED. They do not have the LEAF flag set,
 meaning they could be internal pages, but that is strange since ROOT
 page is at level 1. Also importantly their next XID is set FrozenXid,
 meaning VACUUM FULL was at play. Maybe due to deletes, we reduced the
 hierarchy level or something?

 Hierarchy level is never reduced.

 I'll send you a perl program we wrote for a customer to check for
 strange issues in btrees.  Please give it a spin; it may give you more
 clues.  If you find additional checks to add, please let me know!



I tried to run Alvaro's perl script, but since the index chain is
broken due to the zeroed out page pretty early on, it could not
traverse it very well.

While I rummage around the code more, does anyone have any theories on
the below?

Other peculiarity in the index file is that we found a lot of zeroed
out pages. Blocks from #279 to #518 are all completely zeroed out
without any signs of even a page header. Any ideas on how we can get
so many zeroed out blocks? Apart from the extend code path, I fail to
see any other. And this is an unusually large number of zero pages

Comments appreciated.

Regards,
Nikhils

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Native XML

2011-03-09 Thread Josh Berkus
On 3/9/11 10:11 AM, Bruce Momjian wrote:
 If you are storing xml in an xml column just to get it
 validated, and doing no processing in the DB, then you'd probably
 prefer our current representation.  If you want to build functional
 indexes on xpath expressions, and then run queries that extract data
 using other xpath expressions, you would probably prefer the other
 representation.

Then I think the answer is that we need both data types.  One for
text-XML and one for binary-XML.

For my part, I don't use PostgreSQL's native XML tools for storage of
XML data because the xpath functions are too slow and limited to make PG
useful as an XML database.

-- 
  -- Josh Berkus
 PostgreSQL Experts Inc.
 http://www.pgexperts.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: psql include file using relative path

2011-03-09 Thread Gurjeet Singh
Good question, I hadn't thought of that either, and thinking about it a bit
I think we'd want to keep the current behaviour of \i and provide new
behaviour using a new command.

Say when we are processing a pretty nested file after multiple \ir commands,
a \i relative path file in any of those files should look for that file in
psql's CWD/PWD. That is what the user expects from \i command currently and
I don't think it'd be desirable to break that assumption.

Regards,

On Wed, Mar 9, 2011 at 4:41 PM, David Fetter da...@fetter.org wrote:

 Being able to include relative paths is a really great feature, but
 should it have a UI (well, API) distinct from fixed-path includes?  My
 first instinct is that it shouldn't, but I haven't really thought it
 through thoroughly.

 Cheers,
 David (the tough coughs as he ploughs the dough)
 On Tue, Mar 08, 2011 at 03:19:40PM -0500, Gurjeet Singh wrote:
  Attached patch implements tab completion. It also introduces the
 long-form
  alternative \include_relative for \ir
 
  Regards,
 
  On Tue, Mar 8, 2011 at 2:18 PM, Ibrar Ahmed ibrar.ah...@gmail.com
 wrote:
 
   Gurjeet!
  
   What about tab completion, like in \i command?
  
   On Fri, Feb 25, 2011 at 5:07 AM, Robert Haas robertmh...@gmail.com
   wrote:
On Thu, Feb 24, 2011 at 6:21 PM, Gurjeet Singh 
 singh.gurj...@gmail.com
   wrote:
psql has the ability to execute commands from a file, but if one
   wishes
to develop and provide a modularized set of sql files, then psql is
 not
   very
helpful because the \i command can open file paths either if they
 are
absolute paths or if they are palced correctly relative to psql's
   current
working directory.
   
Attached patch adds a new meta-command to psql, '\ir' that allows
 the
   user
to process files relative to currently processing file.
   
Please add this patch to the currently open CommitFest at:
   
https://commitfest.postgresql.org/action/commitfest_view/open
   
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
   
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
   
  
  
  
   --
  Ibrar Ahmed
  
 
 
 
  --
  Gurjeet Singh
  EnterpriseDB http://www.enterprisedb.com/ Corporation
  The Enterprise PostgreSQL http://www.postgresql.org/ Company


 
  --
  Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
  To make changes to your subscription:
  http://www.postgresql.org/mailpref/pgsql-hackers


 --
 David Fetter da...@fetter.org http://fetter.org/
 Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
 Skype: davidfetter  XMPP: david.fet...@gmail.com
 iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

 Remember to vote!
 Consider donating to Postgres: http://www.postgresql.org/about/donate




-- 
Gurjeet Singh
EnterpriseDB http://www.enterprisedb.com/ Corporation
The Enterprise PostgreSQL http://www.postgresql.org/ Company


Re: [HACKERS] Fwd: index corruption in PG 8.3.13

2011-03-09 Thread Greg Stark
On Wed, Mar 9, 2011 at 11:28 PM, Nikhil Sontakke
nikhil.sonta...@enterprisedb.com wrote:
 Other peculiarity in the index file is that we found a lot of zeroed
 out pages. Blocks from #279 to #518 are all completely zeroed out
 without any signs of even a page header. Any ideas on how we can get
 so many zeroed out blocks? Apart from the extend code path, I fail to
 see any other. And this is an unusually large number of zero pages


What does stat say for the index data file? Are the Size and Blocks
values the same (modulo block size)? Or are these blocks actually not
allocated?

Postgres always forces blocks to be allocated but if they were lost
due to filesystem corruption maybe they're not allocated any more.

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: index corruption in PG 8.3.13

2011-03-09 Thread Nikhil Sontakke

 What does stat say for the index data file? Are the Size and Blocks
 values the same (modulo block size)? Or are these blocks actually not
 allocated?

stat 58401
  File: `58401'
  Size: 4300800 Blocks: 8400   IO Block: 4096   regular file
Device: 801h/2049d  Inode: 13901264Links: 1

modulo 8192 it gives 525 which is the number of blocks that we see.

 Postgres always forces blocks to be allocated but if they were lost
 due to filesystem corruption maybe they're not allocated any more.


Actually these blocks are like a big hole in between. Blocks 279 to
518 are zeroed out and then 519 to 524 is index data. Number 523 is
the one which has been zeroed out too causing all the mayhem.

Regards,
Nikhils

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: psql include file using relative path

2011-03-09 Thread David Fetter
On Wed, Mar 09, 2011 at 07:05:19PM -0500, Gurjeet Singh wrote:
 Good question, I hadn't thought of that either, and thinking about
 it a bit I think we'd want to keep the current behaviour of \i and
 provide new behaviour using a new command.
 
 Say when we are processing a pretty nested file after multiple \ir
 commands, a \i relative path file in any of those files should
 look for that file in psql's CWD/PWD. That is what the user expects
 from \i command currently and I don't think it'd be desirable to
 break that assumption.

I'm not sure I understand.  Stuff that worked before would still work.

Should stuff break when it has a legitimately accessible path in it
just because that path is relative?

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: index corruption in PG 8.3.13

2011-03-09 Thread Alvaro Herrera
Excerpts from Nikhil Sontakke's message of mié mar 09 20:28:19 -0300 2011:

 While I rummage around the code more, does anyone have any theories on
 the below?
 
 Other peculiarity in the index file is that we found a lot of zeroed
 out pages. Blocks from #279 to #518 are all completely zeroed out
 without any signs of even a page header. Any ideas on how we can get
 so many zeroed out blocks? Apart from the extend code path, I fail to
 see any other. And this is an unusually large number of zero pages

Well, if you see the root block child pointers, they go to pages
2, 4-277,522-524.  So pages 278 to 521 seem unused for some reason,
which is rather strange.  But apparently only page 523 is zeroed and
shouldn't be.

It seems hard to believe that there would be 245 unsuccessful attempts
at extending the file.

Page 522 is suspect too ... why does it have a single item?  Note its
LSN is very close to the one on page 521.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] WAL segments pile up during standalone mode

2011-03-09 Thread Bruce Momjian
Tom Lane wrote:
 Alvaro Herrera alvhe...@commandprompt.com writes:
  I admit I have no idea why these guys seem to run into wraparound
  problems so much.
 
  On the other hand, I'm not sure that it would work to try to checkpoint
  during vacuum, because the backend is in a transaction.  Maybe it
  would work to force a checkpoint after each command, and between tables
  in a multi-table vacuum (which is presumably a common thing to do in a
  standalone backend) or something like that?
 
 I really don't care for the idea of standalone mode doing *anything*
 the user didn't explicitly tell it to.  In its role as a disaster
 recovery tool, that's just a recipe for shooting yourself in the foot.
 
 Perhaps this problem would be adequately addressed by documentation,
 ie suggest that when vacuuming very large tables in standalone mode,
 you should issue CHECKPOINT after each one.

I documented that there is no automatic background processing
(checkpoints) in single-user mode.  I did not mention the idea of
running checkpoints manually.  Applied patch attached.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +
diff --git a/doc/src/sgml/ref/postgres-ref.sgml b/doc/src/sgml/ref/postgres-ref.sgml
index d332cba..bd4ae33 100644
*** a/doc/src/sgml/ref/postgres-ref.sgml
--- b/doc/src/sgml/ref/postgres-ref.sgml
*** PostgreSQL documentation
*** 745,751 
  
 para
  Note that the single-user mode server does not provide sophisticated
! line-editing features (no command history, for example).
 /para
   /refsect1
  
--- 745,754 
  
 para
  Note that the single-user mode server does not provide sophisticated
! line-editing features (no command history, for example). 
! Single-User mode also does not do any background processing, like
! automatic checkpoints.
! 
 /para
   /refsect1
  

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Header comments in the recently added files

2011-03-09 Thread Itagaki Takahiro
I found trivial mistakes in the recently added files.
Will they fixed by some automated batches, or by manual?

- Copyright (c) xxx-*2010*, PostgreSQL Global Development Group
  in pg_collation.h, pg_foreign_table.h, basebackup.h, syncrep.h,
  pg_backup_directory.c and auth_delay.c.
- IDENTIFICATION $PostgreSQL$ in pg_collation.h, syncrep.h, and syncrep.c
  Other files has their actual paths in the same place.

-- 
Itagaki Takahiro

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Update of replication/README

2011-03-09 Thread Fujii Masao
On Thu, Mar 10, 2011 at 2:00 AM, Robert Haas robertmh...@gmail.com wrote:
 On Wed, Mar 9, 2011 at 6:11 AM, Fujii Masao masao.fu...@gmail.com wrote:
 The attached patch updates replication/README to reflect current
 walsender/walreceiver behavior. It doesn't include any description
 about sync rep. That would need to be added later.

 Hrm.  What about this hunk?

 -Each walsender allocates an entry from the WalSndCtl array, and advertises
 -there how far it has streamed WAL already. This is used at checkpoints, to
 -avoid recycling WAL that hasn't been streamed to a slave yet. However,
 -that doesn't stop such WAL from being recycled when the connection is not
 -established.
 +Each walsender allocates an entry from the WalSndCtl array, and tracks
 +information about replication progress. User can monitor them via
 +statistics views.

 Is the deleted text not (or no longer) true?

Yes. But, in fact, the deleted text is false in not only 9.1dev but
also 9.0. IIRC,
though my original patch of streaming replication prevented checkpoint from
recycling unsent WAL files, that behavior was cut out and we introduced
wal_keep_segments parameter before release of 9.0. But unfortunately I had
not noticed that text until I read README yesterday...

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Sync Rep v19

2011-03-09 Thread Bruce Momjian
Simon Riggs wrote:
 On Fri, 2011-03-04 at 23:15 +0900, Fujii Masao wrote:
 
  postgres=# SELECT application_name, state, sync_priority, sync_state
  FROM pg_stat_replication;
   application_name |   state   | sync_priority | sync_state
  --+---+---+
   one  | STREAMING | 1 | POTENTIAL
   two  | STREAMING | 2 | SYNC
  (2 rows)
 
 Bug! Thanks.

Is there a reason these status are all upper-case?

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: psql include file using relative path

2011-03-09 Thread Gurjeet Singh
On Wed, Mar 9, 2011 at 7:32 PM, David Fetter da...@fetter.org wrote:

 On Wed, Mar 09, 2011 at 07:05:19PM -0500, Gurjeet Singh wrote:
  Good question, I hadn't thought of that either, and thinking about
  it a bit I think we'd want to keep the current behaviour of \i and
  provide new behaviour using a new command.
 
  Say when we are processing a pretty nested file after multiple \ir
  commands, a \i relative path file in any of those files should
  look for that file in psql's CWD/PWD. That is what the user expects
  from \i command currently and I don't think it'd be desirable to
  break that assumption.

 I'm not sure I understand.  Stuff that worked before would still work.


files present:
~/1.sql
~/package/main.sql
~/package/1.sql

psql's CWD: $HOME
psql executed as: psql -f ~/package/main.sql

Now if we kept \i and \ir separate then if main.sql does

\i 1.sql

it'd read ~/1.sql, which is the current behaviour,

and

\ir 1.sql

would read ~/package/1.sql .

If we folded \ir into \i then what would you want `\i 1.sql` to do? Read
1.sql from $HOME or the one that is main.sql's sibling.



 Should stuff break when it has a legitimately accessible path in it
 just because that path is relative?


Given the above test case, I think it'd be best if we introduced a new
command for  this feature.

Regards,
-- 
Gurjeet Singh
EnterpriseDB http://www.enterprisedb.com/ Corporation
The Enterprise PostgreSQL http://www.postgresql.org/ Company


Re: [HACKERS] How should the waiting backends behave in sync rep?

2011-03-09 Thread Fujii Masao
On Thu, Mar 10, 2011 at 2:06 AM, Robert Haas robertmh...@gmail.com wrote:
 On Tue, Mar 8, 2011 at 10:06 AM, Fujii Masao masao.fu...@gmail.com wrote:
 How should the backends waiting for replication behave when
 synchrnous_standby_names
 is set to '' and the configuration file is reloaded? Now they keep
 waiting for the ACK from the
 standby. But I think that it's more natural for them to get out of the
 wait state and complete
 the transaction in that case. If we'll change them in that way, we
 would no longer need
 something like pg_ctl standalone which I mentioned in another thread. 
 Thought?

 I think so.  Looking at assign_synchronous_standby_names, the
 following code just looks wrong:

        if (doit  list_length(elemlist)  0)
                sync_standbys_defined = true;

 Once sync_standbys_defined becomes true, there's no way for it to ever
 become false again.  That can't be right.  That means that if you
 disable sync rep by zeroing out synchronous_standby_names, backends
 that already existed at the time you made the change will continue to
 act as though sync rep is enabled until they exit.

Yes, that's a bug. Yeah, sync rep currently seems to have many TODO items.
I added some of them in wiki.
http://wiki.postgresql.org/wiki/PostgreSQL_9.1_Open_Items

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: psql include file using relative path

2011-03-09 Thread Andrew Dunstan



On 03/09/2011 09:36 PM, Gurjeet Singh wrote:
If we folded \ir into \i then what would you want `\i 1.sql` to do? 
Read 1.sql from $HOME or the one that is main.sql's sibling.



Should stuff break when it has a legitimately accessible path in it
just because that path is relative?


Given the above test case, I think it'd be best if we introduced a new 
command for  this feature.





I agree there's a good case for the new feature. I think someone 
mentioned tab completion upthread, and that doesn't make so much sense 
to me. This only makes sense nested in a script - in fact if it's not 
called from inside an included script (via -f or \i) it should possibly 
error out (if it already does this I apologise - I haven't looked at the 
patch).


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: psql include file using relative path

2011-03-09 Thread Gurjeet Singh
On Wed, Mar 9, 2011 at 10:07 PM, Andrew Dunstan and...@dunslane.net wrote:


 I agree there's a good case for the new feature. I think someone mentioned
 tab completion upthread, and that doesn't make so much sense to me. This
 only makes sense nested in a script - in fact if it's not called from inside
 an included script (via -f or \i) it should possibly error out (if it
 already does this I apologise - I haven't looked at the patch).


I think \ir can stand on its own. In the patch, the \ir command falls back
to \i behaviour if there's no file being processed currently. So, I think
tab-completion makes sense for this command. And if someone wishes they can
stop using \i altogether and \ir will give them old and new bheaviour
seamlessly.

Regards,
-- 
Gurjeet Singh
EnterpriseDB http://www.enterprisedb.com/ Corporation
The Enterprise PostgreSQL http://www.postgresql.org/ Company


Re: [HACKERS] Header comments in the recently added files

2011-03-09 Thread Robert Haas
On Wed, Mar 9, 2011 at 8:33 PM, Itagaki Takahiro
itagaki.takah...@gmail.com wrote:
 I found trivial mistakes in the recently added files.
 Will they fixed by some automated batches, or by manual?

 - Copyright (c) xxx-*2010*, PostgreSQL Global Development Group
  in pg_collation.h, pg_foreign_table.h, basebackup.h, syncrep.h,
  pg_backup_directory.c and auth_delay.c.
 - IDENTIFICATION $PostgreSQL$ in pg_collation.h, syncrep.h, and syncrep.c
  Other files has their actual paths in the same place.

I think these should be fixed manually.  The first might eventually
get fixed automatically, but perhaps not until 2012, and I'm not sure
the second would ever get fixed automatically.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: psql include file using relative path

2011-03-09 Thread Robert Haas
On Wed, Mar 9, 2011 at 7:32 PM, David Fetter da...@fetter.org wrote:
 On Wed, Mar 09, 2011 at 07:05:19PM -0500, Gurjeet Singh wrote:
 Good question, I hadn't thought of that either, and thinking about
 it a bit I think we'd want to keep the current behaviour of \i and
 provide new behaviour using a new command.

 Say when we are processing a pretty nested file after multiple \ir
 commands, a \i relative path file in any of those files should
 look for that file in psql's CWD/PWD. That is what the user expects
 from \i command currently and I don't think it'd be desirable to
 break that assumption.

 I'm not sure I understand.  Stuff that worked before would still work.

 Should stuff break when it has a legitimately accessible path in it
 just because that path is relative?

You're confused.  The point is whether the path is relative to PWD or
to the directory in which the currently executing script is located.
If you want to allow people to get either interpretation, you need two
commands.

In interactive use, I believe there's no difference between the two.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Header comments in the recently added files

2011-03-09 Thread Itagaki Takahiro
On Thu, Mar 10, 2011 at 12:55, Robert Haas robertmh...@gmail.com wrote:
 On Wed, Mar 9, 2011 at 8:33 PM, Itagaki Takahiro
 itagaki.takah...@gmail.com wrote:
 I found trivial mistakes in the recently added files.
 Will they fixed by some automated batches, or by manual?

 I think these should be fixed manually.  The first might eventually
 get fixed automatically, but perhaps not until 2012, and I'm not sure
 the second would ever get fixed automatically.

OK, I'll do that.

-- 
Itagaki Takahiro

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Added new version of ecpg's parser generator script. This one wa

2011-03-09 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Michael Meskes wrote:
 Added new version of ecpg's parser generator script. This one was written by
 Andy Colson a...@squeakycode.net.

 Uh, why are we not just replacing the old script and allowing git to
 preserve the old version?

Yeah, I was wondering that too.  There seems no very good reason to use
HEAD as a historical repository when we have an SCM that does a very
fine job of archiving old versions.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-09 Thread Fujii Masao
On Thu, Mar 10, 2011 at 12:03 AM, Simon Riggs si...@2ndquadrant.com wrote:
 On Wed, 2011-03-09 at 15:37 +0100, Yeb Havinga wrote:

 The current situation is definately unsafe because it forces people
 that are in this state to do a fast shutdown.. but that fails as well,
 so they are only left with immediate.

I agree with Yeb.

 All the more reason not to change anything, since we disagree.

 The idea is that you're supposed to wait for the standby to come back up
 or do failover. If you shutdown the master its because you are choosing
 to failover.

 Shutting down the master and restarting without failover leads to a
 situation where some sync rep commits are not on both master and
 standby. Making it easier to shutdown encourages that, which I don't
 wish to do, at this stage.

I'm not sure I follow you. The proposed fast shutdown prevents the
backends which have not received the ACK from the standby yet
from returning the success to the client. So even after restarting
the server, there is no data loss from client's point of view. If this is
really unsafe, we *must* forbid immediate shutdown while backend
is waiting for sync rep. Because immediate shutdown creates the
same situation.

What scenario are you concerned?

 We may decide that this is the right approach but I don't wish to rush
 into that decision. I want to have clear agreement about all the changes
 we want and what we call them if we do them. Zero data loss is
 ultimately about users having confidence in us, not about specific
 features. Our disagreements on this patch risk damaging that confidence,
 whoever is right/wrong.

Same as above. I think that it's more problematic to leave the code
as it is. Because smart/fast shutdown can make the server get stuck
until immediate shutdown is requested.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Header comments in the recently added files

2011-03-09 Thread Tom Lane
Itagaki Takahiro itagaki.takah...@gmail.com writes:
 I found trivial mistakes in the recently added files.
 Will they fixed by some automated batches, or by manual?

 - Copyright (c) xxx-*2010*, PostgreSQL Global Development Group
   in pg_collation.h, pg_foreign_table.h, basebackup.h, syncrep.h,
   pg_backup_directory.c and auth_delay.c.
 - IDENTIFICATION $PostgreSQL$ in pg_collation.h, syncrep.h, and syncrep.c
   Other files has their actual paths in the same place.

It might be worth Bruce making another run of his copyright-update
script to fix the former problems.  As for the latter problems,
I have a to-do item to go around and standardize the format of the
header comments a bit better --- I'd like to have a uniform convention
that we put the full path name of each file at the top, and then get
rid of the vestigial IDENTIFICATION sections.  Don't know quite when
I'll get to it though.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [DOCS] Sync rep doc corrections

2011-03-09 Thread Bruce Momjian
Thom Brown wrote:
 On 7 March 2011 22:31, Robert Haas robertmh...@gmail.com wrote:
  On Mon, Mar 7, 2011 at 6:16 AM, Thom Brown t...@linux.com wrote:
  On 7 March 2011 15:27, Thom Brown t...@linux.com wrote:
  I've attached a small patch with a bit of clarification and a typo fix
  in the synchronous_standby_names parameter info.
 
  Okay, I've noticed that the main documentation also needed some fixes,
  so those have been included in this new patch.
 
  I don't think it's necessary to cross-post these emails
 
 I've received conflicting information in the past on this, but I'm
 assuming you're suggesting to send this just to the -docs list in
 future?

I send pure woring changes only to the docs list, and items that are
related to behavior and docs to both.  Does that help?

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] src/test/isolation/specscanner.l is a few bricks shy of a load

2011-03-09 Thread Tom Lane
Somebody needs to brush up their flex-fu:

make[1]: Entering directory `/home/postgres/pgsql/src/test/isolation'
/usr/local/bin/bison  -o specparse.c specparse.y
/usr/local/bin/flex  -o'specscanner.c' specscanner.l
specscanner.l:85: warning, -s option given but default rule can be matched
make[1]: Leaving directory `/home/postgres/pgsql/src/test/isolation'

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Fwd: psql include file using relative path

2011-03-09 Thread David Fetter
On Wed, Mar 09, 2011 at 10:57:53PM -0500, Robert Haas wrote:
 On Wed, Mar 9, 2011 at 7:32 PM, David Fetter da...@fetter.org wrote:
  On Wed, Mar 09, 2011 at 07:05:19PM -0500, Gurjeet Singh wrote:
  Good question, I hadn't thought of that either, and thinking
  about it a bit I think we'd want to keep the current behaviour of
  \i and provide new behaviour using a new command.
 
  Say when we are processing a pretty nested file after multiple
  \ir commands, a \i relative path file in any of those files
  should look for that file in psql's CWD/PWD. That is what the
  user expects from \i command currently and I don't think it'd be
  desirable to break that assumption.
 
  I'm not sure I understand.  Stuff that worked before would still
  work.
 
  Should stuff break when it has a legitimately accessible path in
  it just because that path is relative?
 
 You're confused.  The point is whether the path is relative to PWD
 or to the directory in which the currently executing script is
 located.  If you want to allow people to get either interpretation,
 you need two commands.
 
 In interactive use, I believe there's no difference between the two.

Thanks for clearing that up :)

OK, nothing to see here.  Move along ;)

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] src/test/isolation/specscanner.l is a few bricks shy of a load

2011-03-09 Thread Heikki Linnakangas

On 10.03.2011 06:54, Tom Lane wrote:

Somebody needs to brush up their flex-fu:

make[1]: Entering directory `/home/postgres/pgsql/src/test/isolation'
/usr/local/bin/bison  -o specparse.c specparse.y
/usr/local/bin/flex  -o'specscanner.c' specscanner.l
specscanner.l:85: warning, -s option given but default rule can be matched
make[1]: Leaving directory `/home/postgres/pgsql/src/test/isolation'


Thanks, fixed.

--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Theory of operation of collation patch

2011-03-09 Thread Martijn van Oosterhout
On Tue, Mar 08, 2011 at 08:52:41PM -0500, Tom Lane wrote:
 Another interesting item ... I see that you added a collation field to
 TypeName, apparently on the grounds that the SQL spec includes collation
 in data type.  However, it seems to me that that is nonsense up with
 which we should not put.  data type is basically only used in CAST and

snip

Sounds like a good plan.

Have a nice day,
-- 
Martijn van Oosterhout   klep...@svana.org   http://svana.org/kleptog/
 Patriotism is when love of your own people comes first; nationalism,
 when hate for people other than your own comes first. 
   - Charles de Gaulle


signature.asc
Description: Digital signature