Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-30 Thread Martin Zaun

Heikki Linnakangas wrote:

Andrew Dunstan wrote:

Greg Smith wrote:

On Wed, 23 Jul 2008, Kevin Grittner wrote:

I've been working on an improved archive_command shell script that I 
expect to submit for comments and potential inclusion in the 
documentation as a better base for other people to build on. This is 
one of the options for how it can operate. It would be painful but 
not impossible to convert a subset of that script to run under 
Windows as well, at least enough to cover this particular issue.


A Perl script using the (standard) File::Copy module along with the 
builtin function rename() should be moderately portable. It would to 
be nice not to have to maintain two scripts.


It's also not very nice to require a Perl installation on Windows, just 
for a replacement of Copy. Would a simple .bat script work?


With these avenues to be explored, can the pg_standby patch on the
CommitFest wiki be moved to the Returned with Feedback section?

Regards,
Martin

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-29 Thread Heikki Linnakangas

Andrew Dunstan wrote:



Greg Smith wrote:

On Wed, 23 Jul 2008, Kevin Grittner wrote:


In our scripts we handle this by copying to a temp directory on the
same mount point as the archive directory and doing a mv to the
archive location when the copy is successfully completed. I think
that this even works on Windows. Could that just be documented as a
strong recommendation for the archive script?


This is exactly what I always do. I think the way cp is shown in the 
examples promotes what's really a bad practice for lots of reasons, 
this particular problem being just one of them.


I've been working on an improved archive_command shell script that I 
expect to submit for comments and potential inclusion in the 
documentation as a better base for other people to build on. This is 
one of the options for how it can operate. It would be painful but not 
impossible to convert a subset of that script to run under Windows as 
well, at least enough to cover this particular issue.


A Perl script using the (standard) File::Copy module along with the 
builtin function rename() should be moderately portable. It would to be 
nice not to have to maintain two scripts.


It's also not very nice to require a Perl installation on Windows, just 
for a replacement of Copy. Would a simple .bat script work?


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-28 Thread Heikki Linnakangas

Andrew Dunstan wrote:

Kevin Grittner wrote:
Heikki Linnakangas [EMAIL PROTECTED] wrote: 

We really need a more reliable way of detecting that a file has been
fully copied. 
 
In our scripts we handle this by copying to a temp directory on the

same mount point as the archive directory and doing a mv to the
archive location when the copy is successfully completed.  I think
that this even works on Windows.  Could that just be documented as a
strong recommendation for the archive script?


Needs testing at least. If it does in fact work then we can just adjust 
the docs and be done 


Yeah.

- or maybe provide a .bat file or perl script that 
would work as na archive_command on Windows.


We're not talking about archive_command. We're talking about the thing 
that copies files to the directory that pg_standby polls.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-28 Thread Andrew Dunstan



Heikki Linnakangas wrote:

Andrew Dunstan wrote:


- or maybe provide a .bat file or perl script that would work as na 
archive_command on Windows.


We're not talking about archive_command. We're talking about the thing 
that copies files to the directory that pg_standby polls.


Er, that's what the archive_command is. Look at the pg_standby docs and 
you'll see that that's where we're currently recommending use of windows 
copy. Perhaps you're confusing this with the restore_command?


cheers

andrew

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-28 Thread Heikki Linnakangas

Andrew Dunstan wrote:



Heikki Linnakangas wrote:

Andrew Dunstan wrote:


- or maybe provide a .bat file or perl script that would work as na 
archive_command on Windows.


We're not talking about archive_command. We're talking about the thing 
that copies files to the directory that pg_standby polls.


Er, that's what the archive_command is. Look at the pg_standby docs and 
you'll see that that's where we're currently recommending use of windows 
copy. Perhaps you're confusing this with the restore_command?


Oh, right. I was thinking that archive_command copies the files to an 
archive location, and there's yet another process copying files from 
there to the directory pg_standby polls. But indeed in the simple 
configuration, archive_command is the command that we're interested in.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-28 Thread Greg Smith

On Wed, 23 Jul 2008, Kevin Grittner wrote:


In our scripts we handle this by copying to a temp directory on the
same mount point as the archive directory and doing a mv to the
archive location when the copy is successfully completed.  I think
that this even works on Windows.  Could that just be documented as a
strong recommendation for the archive script?


This is exactly what I always do.  I think the way cp is shown in the 
examples promotes what's really a bad practice for lots of reasons, this 
particular problem being just one of them.


I've been working on an improved archive_command shell script that I 
expect to submit for comments and potential inclusion in the documentation 
as a better base for other people to build on.  This is one of the options 
for how it can operate.  It would be painful but not impossible to convert 
a subset of that script to run under Windows as well, at least enough to 
cover this particular issue.


--
* Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-28 Thread Andrew Dunstan



Greg Smith wrote:

On Wed, 23 Jul 2008, Kevin Grittner wrote:


In our scripts we handle this by copying to a temp directory on the
same mount point as the archive directory and doing a mv to the
archive location when the copy is successfully completed. I think
that this even works on Windows. Could that just be documented as a
strong recommendation for the archive script?


This is exactly what I always do. I think the way cp is shown in the 
examples promotes what's really a bad practice for lots of reasons, 
this particular problem being just one of them.


I've been working on an improved archive_command shell script that I 
expect to submit for comments and potential inclusion in the 
documentation as a better base for other people to build on. This is 
one of the options for how it can operate. It would be painful but not 
impossible to convert a subset of that script to run under Windows as 
well, at least enough to cover this particular issue.





A Perl script using the (standard) File::Copy module along with the 
builtin function rename() should be moderately portable. It would to be 
nice not to have to maintain two scripts.


cheers

andrew

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-25 Thread Simon Riggs

On Fri, 2008-07-25 at 16:31 -0400, Tom Lane wrote:
 Simon Riggs [EMAIL PROTECTED] writes:
  On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote:
  reviewing your patch 
 
  Current status is this:
  * My understanding is that Dave and Andrew (and therefore Simon) think
  the approach proposed here is an acceptable one. Heikki disagrees and
  wants different approach. Perhaps I misunderstand.
  * Patch needs work to complete the proposed approach
  * I'm willing to change the patch, but not able to test it on Windows.
 
 I thought the latest conclusion was that changing the behavior of
 pg_standby itself wouldn't address the problem anyway, and that what we
 need is just a docs patch recommending that people use safe copying
 methods in their scripts that copy to the archive area?

Plus the rest of this patch, which is really very simple.

pg_standby currently waits (on Windows) for the sleep time. We agreed
that this sleep would be on by default, but optional.

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-25 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes:
 On Fri, 2008-07-25 at 16:31 -0400, Tom Lane wrote:
 I thought the latest conclusion was that changing the behavior of
 pg_standby itself wouldn't address the problem anyway, and that what we
 need is just a docs patch recommending that people use safe copying
 methods in their scripts that copy to the archive area?

 Plus the rest of this patch, which is really very simple.

Why?  AFAICT the patch is just a kluge that adds user-visible complexity
without providing a solution that's actually sure to work.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-25 Thread Simon Riggs

On Fri, 2008-07-25 at 16:58 -0400, Tom Lane wrote:
 Simon Riggs [EMAIL PROTECTED] writes:
  On Fri, 2008-07-25 at 16:31 -0400, Tom Lane wrote:
  I thought the latest conclusion was that changing the behavior of
  pg_standby itself wouldn't address the problem anyway, and that what we
  need is just a docs patch recommending that people use safe copying
  methods in their scripts that copy to the archive area?
 
  Plus the rest of this patch, which is really very simple.
 
 Why?  AFAICT the patch is just a kluge that adds user-visible complexity
 without providing a solution that's actually sure to work.

First, I'm not the one objecting to the current behaviour. 

Currently, there is a wait in there that can be removed if you use a
copy utility that sets size after it does a copy. So we agreed to make
it optional (at PGCon).

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Simon Riggs

On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote:

 1. Issues with applying the patch to CVS HEAD:

For me, the patch applies cleanly to CVS HEAD.

I do notice that there are two files standby.sgml and
pgstandby.sgml. I can't see where standby.sgml comes from, but I
haven't created it; perhaps it is a relic of the SGML build process.
I've recreated my source tree since I wrote the patch also. Weird.

I'll redo the patch so it points at pgstandby.sgml, which is the one
thats listed as being in the main source tree.

 2. Missing description for new command-line options in pgstandby.sgml
 
 - no description of the proposed new command-line options -h and -p?

These are done. The patch issues have missed those hunks.

 3. No coding style issues seen
 
 Just one comment: the logic that selects the actual restore command to
 be used has moved from CustomizableInitialize() to main() -- a matter
 of personal taste, perhaps.  But in my view the:
 + the #ifdef WIN32/HAVE_WORKING_LINK logic has become easier to read

Thanks

 4. Issue: missing break in switch, silent override of '-l' argument?
 
 This behaviour has been in there before 

Well spotted. I don't claim to test this for Windows.

 5. Minor wording issue in usage message on new '-p' option
 
 I was wondering if the always in the usage text
  fprintf(stderr,   -p   always uses GNU compatible 'cp' command 
 on all platforms\n);
 is too strong, since multiple restore command options overwrite each
 other, e.g. -p -c applies Windows's copy instead of Gnu's cp.

I was assuming you don't turn the switch off again immediately
afterwards.

 6. Minor code comment suggestion
 
 Unrelated to this patch, I wonder if the code comments on all four
 time-related vars better read seconds instead of amount of time:
 int sleeptime = 5;  /* amount of time to sleep between file 
 checks */
 int holdtime = 0;   /* amount of time to wait once file appears 
 full */
 int waittime = -1;  /* how long we have been waiting, -1 no wait
   * yet */
 int maxwaittime = 0;/* how long are we prepared to wait for? */

As you say, unrelated to the patch.

 7. Question: benefits of separate holdtime option from sleeptime?
 
 Simon Riggs wrote:
   * provide holdtime delay, default 0 (on all platforms)
 
 Going back on the hackers+patches emails and parsing the code
 comments, I'm sorry if I missed that, but I'm not sure I've understood
 the exact tuning benefits that introducing the new holdtime option
 provides over using the existing sleeptime, as it's been the case
 (just on Win32 only).

This is central to the patch, since the complaint was about the delay
introduced by doing that previously.

 8. Unresolved question of implementing now/later a cp replacement

The patch implements what's been agreed. 

I'm not rewriting cp, for reasons already discussed.



Not a comment to you Martin, but it's fairly clear that I'm not
maintaining this correctly for Windows. I've never claimed to have
tested this on Windows, and only included Windows related items as
requested by others. I need to make it clear that I'm not going to
maintain it at all, for Windows. If others wish to report Windows issues
then they can suggest appropriate fixes and test them also.

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Heikki Linnakangas

Simon Riggs wrote:

On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote:

8. Unresolved question of implementing now/later a cp replacement


The patch implements what's been agreed. 


I'm not rewriting cp, for reasons already discussed.

Not a comment to you Martin, but it's fairly clear that I'm not
maintaining this correctly for Windows. I've never claimed to have
tested this on Windows, and only included Windows related items as
requested by others. I need to make it clear that I'm not going to
maintain it at all, for Windows. If others wish to report Windows issues
then they can suggest appropriate fixes and test them also.


Hmm. I just realized that replacing the cp command within pg_standby 
won't help at all. The problem is with the command that copies the files 
*to* the archivelocation that pg_standby polls, not with the copy 
pg_standby does from archivelocation to pg_xlog. And we don't have much 
control over that.


We really need a more reliable way of detecting that a file has been 
fully copied. One simple improvement would be to check the xlp_magic 
field of the last page, though it still wouldn't be bullet-proof.


Do the commands that preallocate the space keep the file exclusively 
locked during the copy? If they do, shouldn't we get an error in trying 
to run the restore copy command, and retry after the 1s sleep in 
RestoreWALFileForRecovery? Though if the archive location is a samba 
mount or something, I guess we can't rely on Windows-style exclusive 
locking.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Kevin Grittner
 Heikki Linnakangas [EMAIL PROTECTED] wrote: 
 
 We really need a more reliable way of detecting that a file has been

 fully copied. 
 
In our scripts we handle this by copying to a temp directory on the
same mount point as the archive directory and doing a mv to the
archive location when the copy is successfully completed.  I think
that this even works on Windows.  Could that just be documented as a
strong recommendation for the archive script?
 
-Kevin

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Andrew Dunstan



Kevin Grittner wrote:
Heikki Linnakangas [EMAIL PROTECTED] wrote: 

 
  

We really need a more reliable way of detecting that a file has been



  
fully copied. 

 
In our scripts we handle this by copying to a temp directory on the

same mount point as the archive directory and doing a mv to the
archive location when the copy is successfully completed.  I think
that this even works on Windows.  Could that just be documented as a
strong recommendation for the archive script?
 



  


Needs testing at least. If it does in fact work then we can just adjust 
the docs and be done - or maybe provide a .bat file or perl script that 
would work as na archive_command on Windows.


cheers

andrew

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Simon Riggs

On Wed, 2008-07-23 at 21:38 +0300, Heikki Linnakangas wrote:
 Simon Riggs wrote:
  On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote:
  8. Unresolved question of implementing now/later a cp replacement
  
  The patch implements what's been agreed. 
  
  I'm not rewriting cp, for reasons already discussed.
  
  Not a comment to you Martin, but it's fairly clear that I'm not
  maintaining this correctly for Windows. I've never claimed to have
  tested this on Windows, and only included Windows related items as
  requested by others. I need to make it clear that I'm not going to
  maintain it at all, for Windows. If others wish to report Windows issues
  then they can suggest appropriate fixes and test them also.
 
 Hmm. I just realized that replacing the cp command within pg_standby 
 won't help at all. The problem is with the command that copies the files 
 *to* the archivelocation that pg_standby polls, not with the copy 
 pg_standby does from archivelocation to pg_xlog. And we don't have much 
 control over that.
 
 We really need a more reliable way of detecting that a file has been 
 fully copied. One simple improvement would be to check the xlp_magic 
 field of the last page, though it still wouldn't be bullet-proof.
 
 Do the commands that preallocate the space keep the file exclusively 
 locked during the copy? If they do, shouldn't we get an error in trying 
 to run the restore copy command, and retry after the 1s sleep in 
 RestoreWALFileForRecovery? Though if the archive location is a samba 
 mount or something, I guess we can't rely on Windows-style exclusive 
 locking.

With respect, I need to refer you back to the my last paragraph above.

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717

2008-07-20 Thread Tatsuo Ishii
 This crashes the backend:
 
 WITH RECURSIVE t(n) AS (
 VALUES (1)
   UNION ALL
 SELECT n+1 FROM t WHERE n  5 ORDER BY 1
 )
 SELECT n FROM t;
 
 apparently because of the  ORDER BY 1

Thanks for the report. I think ORDER BY in this case is useless
anyway. ORDER BY affects (VALUES (1) UNION ALL SELECT n+1 FROM t WHERE
n  5). Since this is a recursive query, value for (VALUES (1) UNION
ALL SELECT n+1 FROM t WHERE n  5) will not be determined until the
recursion stops. So the meaning of ORDER BY is vague. If caller wants
to get the sorted result of the recursion, he could always write:

WITH RECURSIVE t(n) AS (
VALUES (1)
  UNION ALL
SELECT n+1 FROM t WHERE n  5
)
SELECT n FROM t ORDER BY 1;

Thus I think we should avoid this kind of ORDER BY. Probably we should
avoid LIMIT/OFFSET and FOR UPDATE as well. Included patches add the
checking plus minor error messages clarifications. Also I include new
error cases sql.

 ( ORDER BY t.n  will just error out )

 Compiled with:
 
   ./configure \
--prefix=${install_dir} \
--with-pgport=${pgport} \
--quiet  \
--enable-depend  \
--enable-cassert \
--enable-debug   \
--with-openssl
 
 
 hth
 
 Erik Rijkers
 
 
 
 
 
-- UNION
WITH RECURSIVE x(n) AS (SELECT 1 UNION SELECT n+1 FROM x)
  SELECT * FROM x;

-- INTERSECT
WITH RECURSIVE x(n) AS (SELECT 1 INTERSECT SELECT n+1 FROM x)
  SELECT * FROM x;

WITH RECURSIVE x(n) AS (SELECT 1 INTERSECT ALL SELECT n+1 FROM x)
  SELECT * FROM x;

-- EXCEPT
WITH RECURSIVE x(n) AS (SELECT 1 EXCEPT SELECT n+1 FROM x)
  SELECT * FROM x;

WITH RECURSIVE x(n) AS (SELECT 1 EXCEPT ALL SELECT n+1 FROM x)
  SELECT * FROM x;

-- no non-recursive term
WITH RECURSIVE x(n) AS (SELECT n FROM x)
  SELECT * FROM x;

-- recursive term in the left hand side
WITH RECURSIVE x(n) AS (SELECT n FROM x UNION ALL SELECT 1)
  SELECT * FROM x;

CREATE TEMP TABLE y (a int);
INSERT INTO y SELECT generate_series(1, 10);
-- LEFT JOIN
WITH RECURSIVE x(n) AS (SELECT a FROM y WHERE a = 1 UNION ALL SELECT x.n+1 FROM 
y LEFT JOIN x ON x.n = y.a where n  10)
  SELECT * FROM x;

-- RIGHT JOIN
WITH RECURSIVE x(n) AS (SELECT a FROM y WHERE a = 1 UNION ALL SELECT x.n+1 FROM 
x RIGHT JOIN y ON x.n = y.a where n  10)
  SELECT * FROM x;

-- FULL JOIN
WITH RECURSIVE x(n) AS (SELECT a FROM y WHERE a = 1 UNION ALL SELECT x.n+1 FROM 
x FULL JOIN y ON x.n = y.a where n  10)
  SELECT * FROM x;

-- subquery
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x
  WHERE n IN (SELECT * FROM x))
  SELECT * FROM x;

WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x
  WHERE n = 1 AND n IN (SELECT * FROM x))
  SELECT * FROM x;

-- GROUP BY
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x GROUP BY n)
  SELECT * FROM x;

-- HAVING
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x HAVING n  10)
  SELECT * FROM x;

-- aggregate functions
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT count(*) FROM x)
  SELECT * FROM x;

WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT sum(*) FROM x)
  SELECT * FROM x;

-- ORDER BY
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x ORDER BY 1)
  SELECT * FROM x;

-- LIMIT/OFFSET
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x LIMIT 10 OFFSET 1)
  SELECT * FROM x;

-- FOR UPDATE
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x FOR UPDATE)
  SELECT * FROM x;


recursive_query.patch.gz
Description: Binary data

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717

2008-07-20 Thread Tom Lane
Tatsuo Ishii [EMAIL PROTECTED] writes:
 Thus I think we should avoid this kind of ORDER BY. Probably we should
 avoid LIMIT/OFFSET and FOR UPDATE as well.

What of index-optimized SELECT max(...) ?

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717

2008-07-20 Thread Tatsuo Ishii
  Thus I think we should avoid this kind of ORDER BY. Probably we should
  avoid LIMIT/OFFSET and FOR UPDATE as well.
 
 What of index-optimized SELECT max(...) ?

Aggregate functions in a recursive term is prohibited by the
standard. For example, 

WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT max(n) FROM x)
  SELECT * FROM x;

produces an error.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717

2008-07-20 Thread David Fetter
On Mon, Jul 21, 2008 at 08:19:35AM +0900, Tatsuo Ishii wrote:
   Thus I think we should avoid this kind of ORDER BY. Probably we should
   avoid LIMIT/OFFSET and FOR UPDATE as well.
  
  What of index-optimized SELECT max(...) ?
 
 Aggregate functions in a recursive term is prohibited by the
 standard. For example, 
 
 WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT max(n) FROM x)
   SELECT * FROM x;
 
 produces an error.

On the other side of UNION ALL, it's OK, right?  For example,

WITH RECURSIVE x(n) AS (
SELECT max(i) FROM t
UNION ALL
SELECT n+1 FROM x WHERE n  20
)

Cheers,
David.
-- 
David Fetter [EMAIL PROTECTED] http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: [EMAIL PROTECTED]

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717

2008-07-20 Thread Tatsuo Ishii
 On Mon, Jul 21, 2008 at 08:19:35AM +0900, Tatsuo Ishii wrote:
Thus I think we should avoid this kind of ORDER BY. Probably we should
avoid LIMIT/OFFSET and FOR UPDATE as well.
   
   What of index-optimized SELECT max(...) ?
  
  Aggregate functions in a recursive term is prohibited by the
  standard. For example, 
  
  WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT max(n) FROM x)
SELECT * FROM x;
  
  produces an error.
 
 On the other side of UNION ALL, it's OK, right?  For example,
 
 WITH RECURSIVE x(n) AS (
 SELECT max(i) FROM t
 UNION ALL
 SELECT n+1 FROM x WHERE n  20
 )

Yes, aggregate functions in the non-recursive term is allowed by the
standard.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717

2008-07-18 Thread Erik
On Fri, July 18, 2008 03:41, Tatsuo Ishii wrote:
  Here is the lastest WITH RECURSIVE patches against CVS HEAD created by
  Yoshiyuki Asaba and minor corrections by Tatsuo Ishii.

 I tried this patch vs. CVS HEAD used my usual configure option with
 only --with-prefix set, then tried to make, and got:

 make[3]: *** No rule to make target `parse_cte.o', needed by `objfiles.txt'.
  Stop.
 make[3]: Leaving directory `/home/shackle/pgsql/src/backend/parser'
 make[2]: *** [parser-recursive] Error 2
 make[2]: Leaving directory `/home/shackle/pgsql/src/backend'
 make[1]: *** [all] Error 2
 make[1]: Leaving directory `/home/shackle/pgsql/src'
 make: *** [all] Error 2

 Is there something missing?

 Oops. I forgot to include patches against newly added files. Please
 try included patches.



This crashes the backend:

WITH RECURSIVE t(n) AS (
VALUES (1)
  UNION ALL
SELECT n+1 FROM t WHERE n  5 ORDER BY 1
)
SELECT n FROM t;

apparently because of the  ORDER BY 1

( ORDER BY t.n  will just error out )


Compiled with:

./configure \
 --prefix=${install_dir} \
 --with-pgport=${pgport} \
 --quiet  \
 --enable-depend  \
 --enable-cassert \
 --enable-debug   \
 --with-openssl


hth

Erik Rijkers






-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GIN improvements

2008-07-13 Thread Teodor Sigaev

Updated: http://www.sigaev.ru/misc/fast_insert_gin-0.9.gz


need more review of fast_insert yet?  It looked like a number of people
commented on it already.


I still havn't clearness of acceptability for suggested aminsertcleanup  
calling.




--
Teodor Sigaev   E-mail: [EMAIL PROTECTED]
   WWW: http://www.sigaev.ru/

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GIN improvements

2008-07-11 Thread Teodor Sigaev

I've committed the multicolumn one with minor revisions (fix some poor
English in docs and comments, add regression-test coverage).  Do you

Thank you very much.


need more review of fast_insert yet?  It looked like a number of people
commented on it already.
I should modify it to support/synchronize with multicolumn GIN - both patches 
touch the same pieces of code, and I didn't make a single patch to simplify review.


--
Teodor Sigaev   E-mail: [EMAIL PROTECTED]
   WWW: http://www.sigaev.ru/

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GIN improvements

2008-07-09 Thread Neil Conway
On Tue, 2008-07-08 at 14:51 -0400, Tom Lane wrote:
 I'd still like to take a look.

I was tasked with reviewing this for the current commit fest, although
so far I've just been working on grokking the rest of the GIN code. But
if you'd like to review it instead, that's fine with me.

-Neil



-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GIN improvements

2008-07-09 Thread Josh Berkus
Neil,

 I was tasked with reviewing this for the current commit fest, although
 so far I've just been working on grokking the rest of the GIN code. But
 if you'd like to review it instead, that's fine with me.

I have plenty of other stuff I could assign you if you're not needed on 
GIN.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GIN improvements

2008-07-08 Thread Teodor Sigaev
I looked this over and it looks good in general. 

May I think that patch passed review and commit it?

--
Teodor Sigaev   E-mail: [EMAIL PROTECTED]
   WWW: http://www.sigaev.ru/

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GIN improvements

2008-07-08 Thread Tom Lane
Teodor Sigaev [EMAIL PROTECTED] writes:
 I looked this over and it looks good in general. 

 May I think that patch passed review and commit it?

I'd still like to take a look.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Explain XML patch v2

2008-07-04 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes:
 Am Freitag, 4. Juli 2008 schrieb Tom Raney:
 Regarding the XML datum, in order to support that, will all users need  
 to compile with libxml?  Are there any lighter weight solutions to  
 serialize other than libxml?

 You can create XML without libxml.

Seems to me that anyone who wants this feature will probably also want
the existing libxml-based features, so they'll be building with libxml
anyway.  So I'd not be in favor of expending any extra code on a
roll-your-own solution.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1

2008-05-28 Thread Josh Berkus

Tom,


I think this patch is plenty complicated enough without adding useless
restrictive options.


+1 for no additonal GUC options.

--Josh Berkus


--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1

2008-05-25 Thread Hans-Juergen Schoenig

Gregory Stark wrote:

Joshua D. Drake [EMAIL PROTECTED] writes:

  

Couldn't we just have it pay attention to the existing
max_stack_depth?
  

Recursive query does not consume stack. The server enters an infinite
loop without consuming stack. Stack-depth error does not happen.


We could have a separate guc variable which limits the maximum number of
levels of recursive iterations. That might be a useful feature for DBAs that
want to limit their users from issuing an infinite query.
  

statement_timeout :)



Good point.

Though it occurs to me that if you set FETCH_COUNT in psql (or do the
equivalent in your code ) statement_timeout becomes much less useful.

  


i don't think statement_timeout is a good idea at all.
it is not deterministic. depending on the load on the server some 
queries will execute while others fail.

a separate GUC is needed.

   best regards,

  hans



--
Cybertec Schönig  Schönig GmbH
PostgreSQL Solutions and Support
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Tel: +43/1/205 10 35 / 340
www.postgresql-support.de, www.postgresql-support.com


--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1

2008-05-25 Thread Joshua D. Drake

Hans-Juergen Schoenig wrote:

Gregory Stark wrote:

Joshua D. Drake [EMAIL PROTECTED] writes:


  


i don't think statement_timeout is a good idea at all.
it is not deterministic. depending on the load on the server some 
queries will execute while others fail.

a separate GUC is needed.


I don't think we need to add clutter to GUC for something that exists to 
handle the problem at hand. If our real concern is server utilization 
based on user or query resources we need to look at an overall solution 
for that issue not a one off for a single feature.


Sincerely,

Joshua D. Drake



--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1

2008-05-25 Thread Tom Lane
[ catching up on back email ]

Gregory Stark [EMAIL PROTECTED] writes:
 Yoshiyuki Asaba [EMAIL PROTECTED] writes:
 Recursive query does not consume stack. The server enters an infinite
 loop without consuming stack. Stack-depth error does not happen.

 We could have a separate guc variable which limits the maximum number of
 levels of recursive iterations. That might be a useful feature for DBAs that
 want to limit their users from issuing an infinite query.

This whole thread seems to be proposing more and more complicated
solutions for what is really a non-problem given Yoshiyuki-san's point.
It's trivial to construct SQL queries that will run for longer than the
MTBF of your hardware --- eg, forget a few join constraints.  We've
gotten along fine with nothing but query cancel and statement_timeout
for that, and I've seen no one proposing that we need to fix it.
We don't disallow you from writing an infinite loop in plpgsql, either.

I think this patch is plenty complicated enough without adding useless
restrictive options.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1

2008-05-23 Thread David Fetter
On Sat, May 24, 2008 at 05:01:11AM +0900, Yoshiyuki Asaba wrote:
 Hi,
 
 From: David Fetter [EMAIL PROTECTED]
 Subject: Re: [PATCHES] WITH RECURSIVE patch V0.1
 Date: Fri, 23 May 2008 11:26:30 -0700
 
  Where is the new patch?
 
 I will create the revised patch on June.  This is a patch for this
 problem.

Thanks very much :)

Cheers,
David.
-- 
David Fetter [EMAIL PROTECTED] http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: [EMAIL PROTECTED]

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1

2008-05-21 Thread Yoshiyuki Asaba
Hi,

From: David Fetter [EMAIL PROTECTED]
Subject: Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
Date: Mon, 19 May 2008 04:36:30 -0700

   I think it's the other way around. The server should not emit
   infinite number of records.
  
  How about adding new GUC parameter max_recursive_call?
 
 Couldn't we just have it pay attention to the existing
 max_stack_depth?

Recursive query does not consume stack. The server enters an infinite
loop without consuming stack. Stack-depth error does not happen.

Regards,
--
Yoshiyuki Asaba
[EMAIL PROTECTED]

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1

2008-05-21 Thread Joshua D. Drake

  Couldn't we just have it pay attention to the existing
  max_stack_depth?
 
  Recursive query does not consume stack. The server enters an infinite
  loop without consuming stack. Stack-depth error does not happen.
 
 We could have a separate guc variable which limits the maximum number of
 levels of recursive iterations. That might be a useful feature for DBAs that
 want to limit their users from issuing an infinite query.
 

statement_timeout :)

Joshua D. Drake




-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1

2008-05-21 Thread Gregory Stark
Joshua D. Drake [EMAIL PROTECTED] writes:

  Couldn't we just have it pay attention to the existing
  max_stack_depth?
 
  Recursive query does not consume stack. The server enters an infinite
  loop without consuming stack. Stack-depth error does not happen.
 
 We could have a separate guc variable which limits the maximum number of
 levels of recursive iterations. That might be a useful feature for DBAs that
 want to limit their users from issuing an infinite query.

 statement_timeout :)

Good point.

Though it occurs to me that if you set FETCH_COUNT in psql (or do the
equivalent in your code ) statement_timeout becomes much less useful.

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com
  Ask me about EnterpriseDB's On-Demand Production Tuning

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1

2008-05-18 Thread Mark Mielke

Merlin Moncure wrote:

On Sun, May 18, 2008 at 5:22 PM, Zoltan Boszormenyi [EMAIL PROTECTED] wrote:
  

Can we get the rows in tree order, please? I.e. something like this:



Is ordering by tree order defined in the standard when no explicit
order is given?  If not, it probably returns them in the order they
are pulled up, which might be the fastest way


+1 for the fastest way, which I expect to often be find all level 1 
matches, find all level 2 matches, ... If ORDER BY is important, it 
should be specified (although it may be difficult or impossible to 
properly represent ORDER BY for a tree? not sure?) I think most uses of 
recursive require extra client side code to deal with anyways, so only 
relative order is important (order within a particular branch).


There are things I'd like to use this for right now. Currently I use 
plpgsql procedures to implement my own recursion. :-)


Cheers,
mark

--
Mark Mielke [EMAIL PROTECTED]



Re: [HACKERS] [PATCHES] [NOVICE] encoding problems

2008-05-09 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Guillaume Smet wrote:
 I understand your point of view but I really think it's more a
 regression fix than a behavior change.

 If I can get other hackers to say we should backpatch we can consider
 it.

Well, 8.3 is already different from 8.2, and a lot of people will see
this particular aspect of it as a regression.  I'm okay with
backpatching to 8.3 ... though the patch needed rather more testing
than you gave it.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] [NOVICE] encoding problems

2008-05-09 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  Guillaume Smet wrote:
  I understand your point of view but I really think it's more a
  regression fix than a behavior change.
 
  If I can get other hackers to say we should backpatch we can consider
  it.
 
 Well, 8.3 is already different from 8.2, and a lot of people will see
 this particular aspect of it as a regression.  I'm okay with
 backpatching to 8.3 ... though the patch needed rather more testing
 than you gave it.

OK, so Alvaro and Tom want this backpatched.  However, it isn't going to
match 8.2 behavior --- is that OK?

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] [NOVICE] encoding problems

2008-05-09 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Tom Lane wrote:
 Well, 8.3 is already different from 8.2, and a lot of people will see
 this particular aspect of it as a regression.  I'm okay with
 backpatching to 8.3 ... though the patch needed rather more testing
 than you gave it.

 OK, so Alvaro and Tom want this backpatched.  However, it isn't going to
 match 8.2 behavior --- is that OK?

Huh?  8.3 is already hugely different from 8.2 because of the newline
formatting changes.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction

2008-05-02 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes:
 Simon Riggs wrote:
 * We've said here http://www.postgresql.org/docs/faqs.TODO.html that we
 Don't want hints. If that's what we really think, then this patch must
 surely be rejected because its a hint... That isn't my view. I *now*
 think we do need hints of various kinds. 

 cursors_tuple_fraction or OPTIMIZE FOR xxx ROWS isn't the kind of hints 
 we've said no to in the past.

More to the point, I think what we've generally meant by hints is
nonstandard decoration on individual SQL commands (either explicit
syntax or one of those interpret-some-comments kluges).  Simon is
reading the policy in such a way that it would forbid all the planner
cost parameters, which is surely not what is intended.

I see this as being basically another cost parameter, and as such
I don't think it needs more documentation than any of those have.
(Now admittedly you could argue that they could all use a ton more
documentation than they now have, but it's not reasonable to insist
on just this one meeting a different standard.)

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction

2008-05-02 Thread Simon Riggs
On Fri, 2008-05-02 at 12:01 -0400, Tom Lane wrote:
 Heikki Linnakangas [EMAIL PROTECTED] writes:
  Simon Riggs wrote:
  * We've said here http://www.postgresql.org/docs/faqs.TODO.html that we
  Don't want hints. If that's what we really think, then this patch must
  surely be rejected because its a hint... That isn't my view. I *now*
  think we do need hints of various kinds. 
 
  cursors_tuple_fraction or OPTIMIZE FOR xxx ROWS isn't the kind of hints 
  we've said no to in the past.
 
 More to the point, I think what we've generally meant by hints is
 nonstandard decoration on individual SQL commands (either explicit
 syntax or one of those interpret-some-comments kluges).  

Yes, that is definitely an Oracle compatibility thought.

 Simon is
 reading the policy in such a way that it would forbid all the planner
 cost parameters, which is surely not what is intended.

So we're allowed to influence the behaviour of the planner, but just not
by touching the individual statements. OK.

Can we allow a statement like

SET index_weighting = '{{my_index, 0.1},{another_index, 0.5}}'

That would allow us to tell a specific SQL statement that it should use
a cost weighting of 0.1 * normal cost for the my_index index (etc).
SET enable_seqscan = off; is a blunt instrument that can sometimes
achieve the same thing, but insufficiently exact to be really useful.
Many people use that (Sun, in their first published PostgreSQL
benchmark...)

We/I want to make the planner even better, but the above is roughly what
people want while they're waiting for us to get the planner right.

 I see this as being basically another cost parameter, and as such
 I don't think it needs more documentation than any of those have.
 (Now admittedly you could argue that they could all use a ton more
 documentation than they now have, but it's not reasonable to insist
 on just this one meeting a different standard.)

OK, seems fair.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction

2008-05-02 Thread Simon Riggs
On Fri, 2008-05-02 at 12:01 -0400, Tom Lane wrote:

 I see this as being basically another cost parameter, and as such
 I don't think it needs more documentation than any of those have.
 (Now admittedly you could argue that they could all use a ton more
 documentation than they now have, but it's not reasonable to insist
 on just this one meeting a different standard.)

OK, if that's the view then the patch is ready for commit, AFAICS.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction

2008-05-02 Thread Hell, Robert
You're right - that's just a typo in the subject of the post. 
It's called cursor_tuple_fraction in the submitted patch.

Regards,
Robert

-Original Message-
From: Tom Lane [mailto:[EMAIL PROTECTED] 
Sent: Freitag, 02. Mai 2008 22:36
To: Simon Riggs
Cc: Heikki Linnakangas; Hell, Robert; pgsql-patches@postgresql.org;
[EMAIL PROTECTED]
Subject: Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction 

Simon Riggs [EMAIL PROTECTED] writes:
 OK, if that's the view then the patch is ready for commit, AFAICS.

Use of the plural in the name seems a bit odd to me.  Anyone have a
problem with calling it cursor_tuple_fraction instead?

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction

2008-05-02 Thread Simon Riggs
On Fri, 2008-05-02 at 16:36 -0400, Tom Lane wrote:
 Simon Riggs [EMAIL PROTECTED] writes:
  OK, if that's the view then the patch is ready for commit, AFAICS.
 
 Use of the plural in the name seems a bit odd to me.  Anyone have a
 problem with calling it cursor_tuple_fraction instead?

Agreed.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction

2008-05-02 Thread Tom Lane
Hell, Robert [EMAIL PROTECTED] writes:
 You're right - that's just a typo in the subject of the post. 
 It's called cursor_tuple_fraction in the submitted patch.

Ah, I hadn't actually read the patch yet ;-).  As penance for the noise,
I will do so now.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])

2008-04-28 Thread Alvaro Herrera
Tom Lane escribió:
 Alvaro Herrera [EMAIL PROTECTED] writes:
  Brendan Jurd escribi�:
  Here's my attempt to remove the typename field from A_Const.  There
  were a few places (notably flatten_set_variable_args() in guc.c, and
  typenameTypeMod() in parse_type.c) where the code expected to see an
  A_Const with a typename, and I had to adjust for an A_Const within a
  TypeCast.  Nonetheless, there was an overall net reduction of 34 lines
  of code, so I think this was a win.
 
  Do say ... why don't we do away with A_Const altogether and just replace
  it with Value?  After this patch, I don't see what's the difference.
 
 They're logically different things, and after I get done putting a parse
 location field into A_Const, they'll still be physically different too.

Aha.  Are you working from Brendan's patch?  I was going to commit it.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])

2008-04-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes:
 Tom Lane escribió:
 They're logically different things, and after I get done putting a parse
 location field into A_Const, they'll still be physically different too.

 Aha.  Are you working from Brendan's patch?  I was going to commit it.

Sure, go ahead.  I was going to add the parse location at the same time,
but it can perfectly well be done as a separate patch.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])

2008-04-28 Thread Alvaro Herrera
Tom Lane escribió:
 Alvaro Herrera [EMAIL PROTECTED] writes:
  Tom Lane escribió:
  They're logically different things, and after I get done putting a parse
  location field into A_Const, they'll still be physically different too.
 
  Aha.  Are you working from Brendan's patch?  I was going to commit it.
 
 Sure, go ahead.  I was going to add the parse location at the same time,
 but it can perfectly well be done as a separate patch.

I came up with the attached patch.  I added the location bits (although
I am unsure if I got the locations right in the parser), but they are
unused -- figuring out how to use them would take me longer than I can
to spend on this.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Index: src/backend/nodes/copyfuncs.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/nodes/copyfuncs.c,v
retrieving revision 1.392
diff -c -p -r1.392 copyfuncs.c
*** src/backend/nodes/copyfuncs.c	14 Apr 2008 17:05:33 -	1.392
--- src/backend/nodes/copyfuncs.c	28 Apr 2008 20:20:44 -
*** _copyAConst(A_Const *from)
*** 1639,1645 
  			break;
  	}
  
! 	COPY_NODE_FIELD(typename);
  
  	return newnode;
  }
--- 1639,1645 
  			break;
  	}
  
! 	COPY_SCALAR_FIELD(location);
  
  	return newnode;
  }
Index: src/backend/nodes/equalfuncs.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/nodes/equalfuncs.c,v
retrieving revision 1.321
diff -c -p -r1.321 equalfuncs.c
*** src/backend/nodes/equalfuncs.c	14 Apr 2008 17:05:33 -	1.321
--- src/backend/nodes/equalfuncs.c	28 Apr 2008 16:17:02 -
*** _equalParamRef(ParamRef *a, ParamRef *b)
*** 1691,1701 
  static bool
  _equalAConst(A_Const *a, A_Const *b)
  {
! 	if (!equal(a-val, b-val))		/* hack for in-line Value field */
! 		return false;
! 	COMPARE_NODE_FIELD(typename);
! 
! 	return true;
  }
  
  static bool
--- 1691,1697 
  static bool
  _equalAConst(A_Const *a, A_Const *b)
  {
! 	return equal(a-val, b-val);		/* hack for in-line Value field */
  }
  
  static bool
Index: src/backend/nodes/outfuncs.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/nodes/outfuncs.c,v
retrieving revision 1.325
diff -c -p -r1.325 outfuncs.c
*** src/backend/nodes/outfuncs.c	13 Apr 2008 20:51:20 -	1.325
--- src/backend/nodes/outfuncs.c	28 Apr 2008 17:24:10 -
*** _outAConst(StringInfo str, A_Const *node
*** 1945,1951 
  
  	appendStringInfo(str,  :val );
  	_outValue(str, (node-val));
! 	WRITE_NODE_FIELD(typename);
  }
  
  static void
--- 1945,1951 
  
  	appendStringInfo(str,  :val );
  	_outValue(str, (node-val));
! 	WRITE_INT_FIELD(location);
  }
  
  static void
Index: src/backend/parser/gram.y
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/parser/gram.y,v
retrieving revision 2.612
diff -c -p -r2.612 gram.y
*** src/backend/parser/gram.y	14 Apr 2008 17:05:33 -	2.612
--- src/backend/parser/gram.y	28 Apr 2008 19:48:16 -
*** static bool QueryIsRule = FALSE;
*** 91,101 
  
  static Node *makeColumnRef(char *relname, List *indirection, int location);
  static Node *makeTypeCast(Node *arg, TypeName *typename);
! static Node *makeStringConst(char *str, TypeName *typename);
! static Node *makeIntConst(int val);
! static Node *makeFloatConst(char *str);
! static Node *makeAConst(Value *v);
! static A_Const *makeBoolAConst(bool state);
  static FuncCall *makeOverlaps(List *largs, List *rargs, int location);
  static void check_qualified_name(List *names);
  static List *check_func_name(List *names);
--- 91,102 
  
  static Node *makeColumnRef(char *relname, List *indirection, int location);
  static Node *makeTypeCast(Node *arg, TypeName *typename);
! static Node *makeStringConst(char *str, int location);
! static Node *makeStringConstCast(char *str, TypeName *typename, int location);
! static Node *makeIntConst(int val, int location);
! static Node *makeFloatConst(char *str, int location);
! static Node *makeAConst(Value *v, int location);
! static Node *makeBoolAConst(bool state, int location);
  static FuncCall *makeOverlaps(List *largs, List *rargs, int location);
  static void check_qualified_name(List *names);
  static List *check_func_name(List *names);
*** set_rest:	/* Generic SET syntaxes: */
*** 1112,1118 
  	n-kind = VAR_SET_VALUE;
  	n-name = client_encoding;
  	if ($2 != NULL)
! 		n-args = list_make1(makeStringConst($2, NULL));
  	else
  		n-kind = VAR_SET_DEFAULT;
  	$$ = n;
--- 1113,1119 
  	n-kind = VAR_SET_VALUE;
  	n-name = client_encoding;
  	if ($2 != NULL)
! 		n-args = list_make1(makeStringConst($2, @2));
  	else
  		

Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])

2008-04-28 Thread Alvaro Herrera
Alvaro Herrera escribió:
 Tom Lane escribió:
  Alvaro Herrera [EMAIL PROTECTED] writes:
   Tom Lane escribió:
   They're logically different things, and after I get done putting a parse
   location field into A_Const, they'll still be physically different too.
  
   Aha.  Are you working from Brendan's patch?  I was going to commit it.
  
  Sure, go ahead.  I was going to add the parse location at the same time,
  but it can perfectly well be done as a separate patch.
 
 I came up with the attached patch.  I added the location bits (although
 I am unsure if I got the locations right in the parser), but they are
 unused -- figuring out how to use them would take me longer than I can
 to spend on this.

Hmm, I'm now wondering if the location should be added to Value as well,
so that it can be passed down to Const?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])

2008-04-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes:
 I came up with the attached patch.

I wasn't envisioning anything anywhere near this invasive.  We only
need locations on constants in a few contexts, I think.

BTW, you broke _equalAConst() ... it was a bad idea anyway to recast
it on the assumption that there would never again be more than one
field.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])

2008-04-28 Thread Alvaro Herrera
Tom Lane escribió:
 Alvaro Herrera [EMAIL PROTECTED] writes:
  I came up with the attached patch.
 
 I wasn't envisioning anything anywhere near this invasive.  We only
 need locations on constants in a few contexts, I think.

Aha.  OK, I'll commit the original patch and let you deal with the rest
of it :-)

 BTW, you broke _equalAConst() ... it was a bad idea anyway to recast
 it on the assumption that there would never again be more than one
 field.

Oops, sorry, I had intended to revert that part and forgot.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])

2008-04-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes:
 Hmm, I'm now wondering if the location should be added to Value as well,
 so that it can be passed down to Const?

Just for the record, we don't want it in Const.  Parse locations are
only useful in the raw grammar output, mainly because they aren't
helpful unless you still have the original query string laying about.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Avahi support for Postgresql

2008-04-01 Thread Mathias Hasselmann

Am Samstag, den 29.03.2008, 12:25 + schrieb [EMAIL PROTECTED]:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On Sat, Feb 23, 2008 at 01:13:38PM +0100, Mathias Hasselmann wrote:
 
 [...]
 
  Avahi/Bonjour/DNS-SD support[1] is very important, for integrating
  Postgresql with modern desktop environments like OSX, GNOME, KDE: It's
  very convenient to choose active DBMS servers in your local network from
  a list, instead of memorizing cryptic connection parameters. 
 
 [...]
 
  People not wanting DNS-SD support for their server can easily control
  that feature via the --with-avahi configure scripts.
 
 Sorry for a dumb question, but I couldn't figure that out from your
 references [1]..[4]: does that mean that the PostgreSQL server would
 advertise itself on the local net? Or what is the purpose of liking-in
 libavahi into the postmaster?

Yes, that's the purpose.

 Surely one wouldn't want this in a data center? 

Yes, this feature definitely targets small-office use, personal use, DB
developers. Don't know enough about data centers to judge the impact
there, but since Avahi - as used in the patch - announces to the local
network only, the impact sould be small.

Still you can tell Avahi to explicitly announce at a certain, non-local
domain, but this feature is not implemented by the patch. Maybe database
developers in large network environments could make use of such
announcements. It would be trivial to add.

 Is there a possiblity to disable that at run time?

The feature is disabled by default. As long as you do not specify a
zeroconf_name in your configuration file, nothing happens. This is the
same behavior as established by the Bonjour code.

Ciao,
Mathias
-- 
Mathias Hasselmann [EMAIL PROTECTED]
http://www.openismus.com/ - We can get it done.


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Avahi support for Postgresql

2008-04-01 Thread tomas
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tue, Apr 01, 2008 at 09:35:56AM +0200, Mathias Hasselmann wrote:
 Am Samstag, den 29.03.2008, 12:25 + schrieb [EMAIL PROTECTED]:
 [...]
  Sorry for a dumb question, but I couldn't figure that out from your
  references [1]..[4]: does that mean that the PostgreSQL server would
  advertise itself on the local net? Or what is the purpose of liking-in
  libavahi into the postmaster?
 
 Yes, that's the purpose.
 
  Surely one wouldn't want this in a data center? 
 
 Yes, this feature definitely targets small-office use, personal use, DB
 developers [...]
 Still you can tell Avahi to explicitly announce at a certain, non-local
 domain, but this feature is not implemented by the patch. Maybe database
 developers in large network environments could make use of such
 announcements. It would be trivial to add.

Personally, I'be rather scared than delighted ;-)

  Is there a possiblity to disable that at run time?
 
 The feature is disabled by default. As long as you do not specify a
 zeroconf_name in your configuration file, nothing happens. This is the
 same behavior as established by the Bonjour code.

Thanks, good to know.

Isn't there a less-intrusive option to linking a lib into each and every
possible server, like a config file in which to put what is to be announced?

Regards
- -- tomás
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFH8iRFBcgs9XrR2kYRAmJ0AJkB7MkxfYI0nVa4RqHVEV1HYjz41gCdEgWz
YQ2T4Y/xfoLRF4D6hMLbpEk=
=Goho
-END PGP SIGNATURE-

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Avahi support for Postgresql

2008-04-01 Thread Mathias Hasselmann

Am Dienstag, den 01.04.2008, 12:02 + schrieb [EMAIL PROTECTED]:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On Tue, Apr 01, 2008 at 09:35:56AM +0200, Mathias Hasselmann wrote:
  Am Samstag, den 29.03.2008, 12:25 + schrieb [EMAIL PROTECTED]:
  [...]
   Sorry for a dumb question, but I couldn't figure that out from your
   references [1]..[4]: does that mean that the PostgreSQL server would
   advertise itself on the local net? Or what is the purpose of liking-in
   libavahi into the postmaster?
  
  Yes, that's the purpose.
  
   Surely one wouldn't want this in a data center? 
  
  Yes, this feature definitely targets small-office use, personal use, DB
  developers [...]
  Still you can tell Avahi to explicitly announce at a certain, non-local
  domain, but this feature is not implemented by the patch. Maybe database
  developers in large network environments could make use of such
  announcements. It would be trivial to add.
 
 Personally, I'be rather scared than delighted ;-)

So in data centers you don't even trust the machines in your broadcast
domain?

   Is there a possiblity to disable that at run time?
  
  The feature is disabled by default. As long as you do not specify a
  zeroconf_name in your configuration file, nothing happens. This is the
  same behavior as established by the Bonjour code.
 
 Thanks, good to know.
 
 Isn't there a less-intrusive option to linking a lib into each and every
 possible server, like a config file in which to put what is to be announced?

You could directly talk to the D-Bus interface of Avahi. libavahi-client
just is a convenience wrapper. Well, but this route will be much more
cumbersome.

One other route is calling avahi-publish-service on startup and killing
it on shutdown, but: avahi-publish-service really only exists for
demonstration purposes and doesn't handle service name collisions for
instance. I don't believe that a high-profile application like
Postgresql should rely on low-quality hacks, like invoking educational
demo programs.

Ciao,
Mathias
-- 
Mathias Hasselmann [EMAIL PROTECTED]
http://www.openismus.com/ - We can get it done.


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Avahi support for Postgresql

2008-04-01 Thread tomas
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tue, Apr 01, 2008 at 05:07:31PM +0200, Mathias Hasselmann wrote:
[...]
  Personally, I'be rather scared than delighted ;-)
 
 So in data centers you don't even trust the machines in your broadcast
 domain?

Kind of. Put it another way: never have services running you don't use.

[...]

  Isn't there a less-intrusive option to linking a lib into each and every
  possible server, like a config file in which to put what is to be announced?
 
 You could directly talk to the D-Bus interface of Avahi. libavahi-client
 just is a convenience wrapper. Well, but this route will be much more
 cumbersome.

So this goes through the D-Bus. Makes kind of sense. Thanks for the
enlightenment.

 One other route is calling avahi-publish-service on startup and killing
 it on shutdown, but: avahi-publish-service really only exists for
 demonstration purposes and doesn't handle service name collisions for
 instance. I don't believe that a high-profile application like
 Postgresql should rely on low-quality hacks, like invoking educational
 demo programs.

Unelegant as it might seem -- this solution still affords a lot more
when it comes to separation of concerns. I'm still a bit wary at the
prospect that each and every daemon evolves into a huge fuzzball
linked to all conceivable service-lets with a multitude of funny
side-effects (remember tcpwrappers?).

Of course, you can always disable this at compile time, but let's face
it: with the predominance of binary distribs, the path of least
resistance will be to put up with whatever strange side-effects.

I would really prefer a more loosely coupled system.

Regards
- -- tomás
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFH8lYJBcgs9XrR2kYRAmDJAJ4jWKYkhUfKEAIaZVnIbAAEqJF2AwCfS/6D
4rH9OoY7wjia7h1cuk5CjZI=
=AF1W
-END PGP SIGNATURE-

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Avahi support for Postgresql

2008-04-01 Thread Murray Cumming
On Tue, 2008-04-01 at 15:34 +, [EMAIL PROTECTED] wrote:
 I would really prefer a more loosely coupled system.

The functionality will be much the same. The implementation would be
more difficult and obscure and there would be more points of failure and
more things to configure, but it wouldn't remove much risk, I think.

Anyway, this feature is already in Postgres when it's built for MacOS X.
So this decision seems to have been made already, at least for that
platform.

-- 
[EMAIL PROTECTED]
www.murrayc.com
www.openismus.com


-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Avahi support for Postgresql

2008-03-29 Thread tomas
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Sat, Feb 23, 2008 at 01:13:38PM +0100, Mathias Hasselmann wrote:

[...]

 Avahi/Bonjour/DNS-SD support[1] is very important, for integrating
 Postgresql with modern desktop environments like OSX, GNOME, KDE: It's
 very convenient to choose active DBMS servers in your local network from
 a list, instead of memorizing cryptic connection parameters. 

[...]

 People not wanting DNS-SD support for their server can easily control
 that feature via the --with-avahi configure scripts.

Sorry for a dumb question, but I couldn't figure that out from your
references [1]..[4]: does that mean that the PostgreSQL server would
advertise itself on the local net? Or what is the purpose of liking-in
libavahi into the postmaster?

Surely one wouldn't want this in a data center? Is there a possiblity to
disable that at run time?

Thanks for any insights

- -- tomás
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFH7jUxBcgs9XrR2kYRAsfOAJ0Ulfydo3G1kzQo3HTOdjtswA1A2gCfYYyL
VHprJ63unId85/Iht4ha2RE=
=get0
-END PGP SIGNATURE-

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-19 Thread Peter Eisentraut
Zdenek Kotala wrote:
 But how it was mentioned in this thread maybe
 somethink like this CREATE TABLESPACE name LOCATION '/my/location'
 SEGMENTS 10GB should good solution. If segments is not mentioned then
 default value is used.

I think you would need a tool to resegmentize a table or tablespace offline, 
usable for example when recovering a backup.

Also, tablespace configuration information is of course also stored in a 
table.  pg_tablespace probably won't become large, but it would probably 
still need to be special-cased, along with other system catalogs perhaps.

An then, how to coordindate offline resegmenting and online tablespace 
operations in a crash-safe way?

Another factor I just thought of is that tar, commonly used as part of a 
backup procedure, can on some systems only handle files up to 8 GB in size.  
There are supposed to be newer formats that can avoid that restriction, but 
it's not clear how widely available these are and what the incantation is to 
get at them.  Of course we don't use tar directly, but if we ever make large 
segments the default, we ought to provide some clear advice for the user on 
how to make their backups.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-19 Thread Martijn van Oosterhout
On Wed, Mar 19, 2008 at 09:38:12AM +0100, Peter Eisentraut wrote:
 Another factor I just thought of is that tar, commonly used as part of a 
 backup procedure, can on some systems only handle files up to 8 GB in size.  
 There are supposed to be newer formats that can avoid that restriction, but 
 it's not clear how widely available these are and what the incantation is to 
 get at them.  Of course we don't use tar directly, but if we ever make large 
 segments the default, we ought to provide some clear advice for the user on 
 how to make their backups.

By my reading, GNU tar handles larger files and no-one else (not even a
POSIX standard tar) can...

Have a nice day,
-- 
Martijn van Oosterhout   [EMAIL PROTECTED]   http://svana.org/kleptog/
 Please line up in a tree and maintain the heap invariant while 
 boarding. Thank you for flying nlogn airlines.


signature.asc
Description: Digital signature


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-19 Thread Kenneth Marshall
On Wed, Mar 19, 2008 at 10:51:12AM +0100, Martijn van Oosterhout wrote:
 On Wed, Mar 19, 2008 at 09:38:12AM +0100, Peter Eisentraut wrote:
  Another factor I just thought of is that tar, commonly used as part of a 
  backup procedure, can on some systems only handle files up to 8 GB in size. 
   
  There are supposed to be newer formats that can avoid that restriction, but 
  it's not clear how widely available these are and what the incantation is 
  to 
  get at them.  Of course we don't use tar directly, but if we ever make 
  large 
  segments the default, we ought to provide some clear advice for the user on 
  how to make their backups.
 
 By my reading, GNU tar handles larger files and no-one else (not even a
 POSIX standard tar) can...
 
 Have a nice day,
 -- 
 Martijn van Oosterhout   [EMAIL PROTECTED]   http://svana.org/kleptog/
  Please line up in a tree and maintain the heap invariant while 
  boarding. Thank you for flying nlogn airlines.

The star program written by Joerg Schilling is a very well written
POSIX compatible tar program that can easily handle files larger than
8GB. It is another backup option.

Cheers,
Ken

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Peter Eisentraut
Tom Lane wrote:
 I think this needs to be treated as experimental until it's got a few
 more than zero miles under its belt.

OK, then maybe we should document that.

 I wouldn't be too surprised to 
 find that we have to implement it as a run-time switch instead of
 compile-time, in order to not fail miserably when somebody sticks a
 tablespace on an archaic filesystem.

Yes, that sounds quite useful.  Let's wait and see what happens.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Zeugswetter Andreas OSB SD

  Why is this not the default when supported?  I am wondering both
from the 
  point of view of the user, and in terms of development direction.
 
 Also it would get more buildfarm coverage if it were default.  If it
 breaks something we'll notice earlier.

No we don't, because the buildfarm does not test huge files.

Andreas

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Larry Rosenman

On Mon, 10 Mar 2008, Tom Lane wrote:


Peter Eisentraut [EMAIL PROTECTED] writes:

Tom Lane wrote:

Applied with minor corrections.



Why is this not the default when supported?


Fear.

Maybe eventually, but right now I think it's too risky.

One point that I already found out the hard way is that sizeof(off_t) = 8
does not guarantee the availability of largefile support; there can also
be filesystem-level constraints, and perhaps other things we know not of
at this point.


Just to note an additional filesystem that will need special action...
The VxFS filesystem has a largefiles option, per filesystem.  At least that
was the case on SCO UnixWare (No, I no longer run it).

LER




regards, tom lane




--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 512-248-2683 E-Mail: [EMAIL PROTECTED]
US Mail: 430 Valona Loop, Round Rock, TX 78681-3893

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes:
 Tom Lane wrote:
 I think this needs to be treated as experimental until it's got a few
 more than zero miles under its belt.

 OK, then maybe we should document that.

Agreed, but at this point we don't even know what hazards we need to
document.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Tom Lane
Zeugswetter Andreas OSB SD [EMAIL PROTECTED] writes:
 Exactly, e.g. AIX is one of those. jfs (not the newer jfs2) has an
 option
 to enable large files, which is not the default and cannot be changed
 post crfs.
 And even if it is enabled, jfs has a 64 Gb filesize limit !
 Anybody know others that support large but not huge files ?

Yeah, HPUX 10 is similar --- 128GB hard maximum.  It does say you
can convert an existing filesystem to largefile support, but it has
to be unmounted.

These examples suggest that maybe what we want is not so much a no
segments ever mode as a segment size larger than 1GB.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Zdenek Kotala

Tom Lane napsal(a):

Zeugswetter Andreas OSB SD [EMAIL PROTECTED] writes:

Exactly, e.g. AIX is one of those. jfs (not the newer jfs2) has an
option
to enable large files, which is not the default and cannot be changed
post crfs.
And even if it is enabled, jfs has a 64 Gb filesize limit !
Anybody know others that support large but not huge files ?


Yeah, HPUX 10 is similar --- 128GB hard maximum.  It does say you
can convert an existing filesystem to largefile support, but it has
to be unmounted.

These examples suggest that maybe what we want is not so much a no
segments ever mode as a segment size larger than 1GB.


Patch allows to use bigger than 2/4GB segment files and it is possible 
changed it in source file. But how it was mentioned in this thread maybe 
somethink like this CREATE TABLESPACE name LOCATION '/my/location' 
SEGMENTS 10GB should good solution. If segments is not mentioned then 
default value is used.



Zdenek

PS: ZFS is happy with 2^64bit size and UFS has 1TB file size limit 
(depends on solaris version)


--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Tom Lane
Zdenek Kotala [EMAIL PROTECTED] writes:
 Tom Lane napsal(a):
 These examples suggest that maybe what we want is not so much a no
 segments ever mode as a segment size larger than 1GB.

 PS: ZFS is happy with 2^64bit size and UFS has 1TB file size limit 
 (depends on solaris version)

So even on Solaris, no segments ever is actually a pretty awful idea.
As it stands, the code would fail on tables  1TB.

I'm thinking we need to reconsider this patch.  Rather than disabling
segmentation altogether, we should see it as allowing use of segments
larger than 1GB.  I suggest that we ought to just flat rip out the non
segmenting code paths in md.c, and instead look into what segment sizes
are appropriate on different platforms.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Bruce Momjian
Tom Lane wrote:
 Zdenek Kotala [EMAIL PROTECTED] writes:
  Tom Lane napsal(a):
  These examples suggest that maybe what we want is not so much a no
  segments ever mode as a segment size larger than 1GB.
 
  PS: ZFS is happy with 2^64bit size and UFS has 1TB file size limit 
  (depends on solaris version)
 
 So even on Solaris, no segments ever is actually a pretty awful idea.
 As it stands, the code would fail on tables  1TB.
 
 I'm thinking we need to reconsider this patch.  Rather than disabling
 segmentation altogether, we should see it as allowing use of segments
 larger than 1GB.  I suggest that we ought to just flat rip out the non
 segmenting code paths in md.c, and instead look into what segment sizes
 are appropriate on different platforms.

Agreed.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Zdenek Kotala

Tom Lane napsal(a):

Zdenek Kotala [EMAIL PROTECTED] writes:

Tom Lane napsal(a):

These examples suggest that maybe what we want is not so much a no
segments ever mode as a segment size larger than 1GB.


PS: ZFS is happy with 2^64bit size and UFS has 1TB file size limit 
(depends on solaris version)


So even on Solaris, no segments ever is actually a pretty awful idea.
As it stands, the code would fail on tables  1TB.

I'm thinking we need to reconsider this patch.  Rather than disabling
segmentation altogether, we should see it as allowing use of segments
larger than 1GB.  I suggest that we ought to just flat rip out the non
segmenting code paths in md.c, and instead look into what segment sizes
are appropriate on different platforms.


Yes, agree. It seems only ZFS is OK at this moment and if somebody sets 
32TB he gets nonsegment mode anyway. I looked into posix standard and 
there is useful function which can be used. See


http://www.opengroup.org/onlinepubs/009695399/functions/pathconf.html

Maybe we can put additional test into configure and collect appropriate 
data from buildfarm.


I think current patch could stay in CVS and I will rip out non segment 
code path in a new patch.


Zdenek

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Tom Lane
Zdenek Kotala [EMAIL PROTECTED] writes:
 I think current patch could stay in CVS and I will rip out non segment 
 code path in a new patch.

Sure, I feel no need to revert what's applied.  Have at it.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Peter Eisentraut
Zdenek Kotala wrote:
 Yes, agree. It seems only ZFS is OK at this moment and if somebody sets
 32TB he gets nonsegment mode anyway.

Surely if you set the segment size to INT64_MAX, you will get nonsegmented 
behavior anyway, so two code paths might not be necessary at all.

 I looked into posix standard and 
 there is useful function which can be used. See

 http://www.opengroup.org/onlinepubs/009695399/functions/pathconf.html

 Maybe we can put additional test into configure and collect appropriate
 data from buildfarm.

It might be good to just check first if it returns realistic values for the 
example cases that have been mentioned.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-11 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes:
 Zdenek Kotala wrote:
 Maybe we can put additional test into configure and collect appropriate
 data from buildfarm.

 It might be good to just check first if it returns realistic values for the 
 example cases that have been mentioned.

Yeah, please just make up a ten-line C program that prints the numbers
you want, and post it on -hackers for people to try.  If manual testing
says that it's printing useful numbers, then it would be time enough to
think about how to get it into the buildfarm.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-10 Thread Tom Lane
Zdenek Kotala [EMAIL PROTECTED] writes:
 There is latest version of nonsegment support patch. I changed 
 LET_OS_MANAGE_FILESIZE to USE_SEGMENTED_FILES and I added 
 -disable-segmented-files switch to configure. I kept tuplestore behavior 
 and it still split file in both mode.

Applied with minor corrections.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-10 Thread Peter Eisentraut
Tom Lane wrote:
 Zdenek Kotala [EMAIL PROTECTED] writes:
  There is latest version of nonsegment support patch. I changed
  LET_OS_MANAGE_FILESIZE to USE_SEGMENTED_FILES and I added
  -disable-segmented-files switch to configure. I kept tuplestore behavior
  and it still split file in both mode.

 Applied with minor corrections.

Why is this not the default when supported?  I am wondering both from the 
point of view of the user, and in terms of development direction.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-10 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes:
 Tom Lane wrote:
 Applied with minor corrections.

 Why is this not the default when supported?

Fear.

Maybe eventually, but right now I think it's too risky.

One point that I already found out the hard way is that sizeof(off_t) = 8
does not guarantee the availability of largefile support; there can also
be filesystem-level constraints, and perhaps other things we know not of
at this point.

I think this needs to be treated as experimental until it's got a few
more than zero miles under its belt.  I wouldn't be too surprised to
find that we have to implement it as a run-time switch instead of
compile-time, in order to not fail miserably when somebody sticks a
tablespace on an archaic filesystem.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-10 Thread Alvaro Herrera
Peter Eisentraut wrote:
 Tom Lane wrote:
  Zdenek Kotala [EMAIL PROTECTED] writes:
   There is latest version of nonsegment support patch. I changed
   LET_OS_MANAGE_FILESIZE to USE_SEGMENTED_FILES and I added
   -disable-segmented-files switch to configure. I kept tuplestore behavior
   and it still split file in both mode.
 
  Applied with minor corrections.
 
 Why is this not the default when supported?  I am wondering both from the 
 point of view of the user, and in terms of development direction.

Also it would get more buildfarm coverage if it were default.  If it
breaks something we'll notice earlier.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)

2008-03-10 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes:
 Also it would get more buildfarm coverage if it were default.  If it
 breaks something we'll notice earlier.

Since nothing the regression tests do even approach 1GB, the odds that
the buildfarm will notice problems are approximately zero.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [HACKERS] [PATCHES] 2WRS [WIP]

2008-02-27 Thread manolo.espa

Referring to tuplesort.c   andtuplestore.c

BACKGROUND: Starting from dumptuples()  [ tuplesort.c ] write functions move 
the tuple from a buffer to another in order to finally write it in a logical 
tape. Is there a way (even the most inefficient way) to use current 
read/write functions provided by PostgreSQL in order to retrieve the first 
tuple of a certain run while performing External Sorting?


NOTE: I need the first tuple in order to manipulate the whole corresponding 
run, tuple by tuple since they are written sequentially in a run.


Thanks for your attention.
Regards, Manolo.


--
From: [EMAIL PROTECTED]
Sent: Tuesday, February 26, 2008 4:10 PM
To: Jaime Casanova [EMAIL PROTECTED]; [EMAIL PROTECTED]
Cc: Decibel! [EMAIL PROTECTED]; David Fetter [EMAIL PROTECTED]; 
pgsql-patches@postgresql.org; [EMAIL PROTECTED]

Subject: Re: [HACKERS] [PATCHES] 2WRS [WIP]


For the joy of all of you: that's the correct WIP patch.
At the moment it only tries to create runs uding two heaps. Hope you can
help me with writing those runs on tapes.

I'd be very pleased to give you more details.

Thenks for your time.
Regards, Manolo.


--
From: Jaime Casanova [EMAIL PROTECTED]
Sent: Friday, February 22, 2008 5:30 AM
To: [EMAIL PROTECTED]
Cc: Decibel! [EMAIL PROTECTED]; Manolo _ [EMAIL PROTECTED];
David Fetter [EMAIL PROTECTED]; pgsql-patches@postgresql.org;
[EMAIL PROTECTED]
Subject: Re: [HACKERS] [PATCHES] 2WRS [WIP]


On Thu, Feb 21, 2008 at 6:44 AM,  [EMAIL PROTECTED] wrote:

Hi.

That's the last release and refers to 8.3.0 and not to 8.2.5 as before.
Hope
you can tell me if I created it correctly please.



no, it doesn't...


! /* GUC variables */
  #ifdef TRACE_SORT
  bool trace_sort = false;
  #endif
- #ifdef DEBUG_BOUNDED_SORT
- bool optimize_bounded_sort = true;
- #endif


it's seems you're removing something added in 8.3

--
regards,
Jaime Casanova

Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs and the universe trying
to produce bigger and better idiots.
So far, the universe is winning.
Richard Cook

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match



---(end of broadcast)---
TIP 4: Have you searched our list archives?

  http://archives.postgresql.org


Re: [HACKERS] [PATCHES] 2WRS [WIP]

2008-02-26 Thread mac_man2005

For the joy of all of you: that's the correct WIP patch.
At the moment it only tries to create runs uding two heaps. Hope you can 
help me with writing those runs on tapes.


I'd be very pleased to give you more details.

Thenks for your time.
Regards, Manolo.

--
From: Jaime Casanova [EMAIL PROTECTED]
Sent: Friday, February 22, 2008 5:30 AM
To: [EMAIL PROTECTED]
Cc: Decibel! [EMAIL PROTECTED]; Manolo _ [EMAIL PROTECTED]; 
David Fetter [EMAIL PROTECTED]; pgsql-patches@postgresql.org; 
[EMAIL PROTECTED]

Subject: Re: [HACKERS] [PATCHES] 2WRS [WIP]


On Thu, Feb 21, 2008 at 6:44 AM,  [EMAIL PROTECTED] wrote:

Hi.

That's the last release and refers to 8.3.0 and not to 8.2.5 as before. 
Hope

you can tell me if I created it correctly please.



no, it doesn't...


! /* GUC variables */
  #ifdef TRACE_SORT
  bool trace_sort = false;
  #endif
- #ifdef DEBUG_BOUNDED_SORT
- bool optimize_bounded_sort = true;
- #endif


it's seems you're removing something added in 8.3

--
regards,
Jaime Casanova

Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs and the universe trying
to produce bigger and better idiots.
So far, the universe is winning.
Richard Cook

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


tuplesort.patch
Description: Binary data

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] [PATCHES] 2WRS [WIP]

2008-02-26 Thread manolo.espa

For the joy of all of you: that's the correct WIP patch.
At the moment it only tries to create runs uding two heaps. Hope you can
help me with writing those runs on tapes.

I'd be very pleased to give you more details.

Thenks for your time.
Regards, Manolo.


--
From: Jaime Casanova [EMAIL PROTECTED]
Sent: Friday, February 22, 2008 5:30 AM
To: [EMAIL PROTECTED]
Cc: Decibel! [EMAIL PROTECTED]; Manolo _ [EMAIL PROTECTED]; 
David Fetter [EMAIL PROTECTED]; pgsql-patches@postgresql.org; 
[EMAIL PROTECTED]

Subject: Re: [HACKERS] [PATCHES] 2WRS [WIP]


On Thu, Feb 21, 2008 at 6:44 AM,  [EMAIL PROTECTED] wrote:

Hi.

That's the last release and refers to 8.3.0 and not to 8.2.5 as before. 
Hope

you can tell me if I created it correctly please.



no, it doesn't...


! /* GUC variables */
  #ifdef TRACE_SORT
  bool trace_sort = false;
  #endif
- #ifdef DEBUG_BOUNDED_SORT
- bool optimize_bounded_sort = true;
- #endif


it's seems you're removing something added in 8.3

--
regards,
Jaime Casanova

Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs and the universe trying
to produce bigger and better idiots.
So far, the universe is winning.
Richard Cook

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


tuplesort.patch
Description: Binary data

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] [PATCHES] Avahi support for Postgresql

2008-02-25 Thread Alvaro Herrera
Mathias Hasselmann wrote:

 The patches were in my initial mail, but now I've also uploaded them to
 my personal site for convenience:
 
   http://taschenorakel.de/files/pgsql-avahi-support/

Hmm, a quick look at the third patch reveals that it is using the
threaded Avahi client.  That's a showstopper.  Please consider using
some other approach -- writing our own handlers for AvahiPoll would seem
apropos.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Fixes for MONEY type using locale

2007-11-26 Thread Bruce Momjian
Tom Lane wrote:
 Joshua D. Drake [EMAIL PROTECTED] writes:
  Well if we are going to continue to support money (which I am against)
  we should support the casting to numeric as that is by far a more
  common implementation of money and we will have mixed environments.
 
 So, you don't use MONEY, and you don't want to, but nonetheless you
 know better than the people who do use MONEY what they need.
 
 Aside from the semantic-gap issue, there is the point that providing
 a cast might actually mask application errors.  I can well imagine
 cases where one of the reasons for using MONEY is *exactly* that it's
 not a plain number or easily convertible to one.

Right.  I am not thinking of an auto-cast but rather give people _some_
way to cast to/from MONEY, which is what the TODO says:

* Allow MONEY to be cast to/from other numeric data types

Even when we have multiple currency designations I would think people
would need a way to cast.  I am trying to anticpate how MONEY will be
used.  You are right we don't have any field requests yet, but I am
expecting them.

I have added documentation on the issues of casting to/from MONEY; 
patch attached and applied.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/datatype.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/datatype.sgml,v
retrieving revision 1.217
diff -c -c -r1.217 datatype.sgml
*** doc/src/sgml/datatype.sgml	21 Nov 2007 04:01:37 -	1.217
--- doc/src/sgml/datatype.sgml	27 Nov 2007 05:45:13 -
***
*** 842,847 
--- 842,855 
  floating-point literals, as well as quotetypical/quote
  currency formatting, such as literal'$1,000.00'/literal.
  Output is generally in the latter form but depends on the locale.
+ Non-quoted numeric values can be converted to typemoney/type by
+ casting the numeric value to typetext/type and then
+ typemoney/type:
+ programlisting
+ SELECT 1234::text::money;
+ /programlisting
+ There is no simple way of doing the reverse;  casting a typemoney/type value to a
+ numeric type.
 /para
  
 para

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit

2007-06-08 Thread Alvaro Herrera
Alvaro Herrera wrote:
 Matthew T. O'Connor wrote:

  Can you make 0 and -1 both valid disabled values?  That way it will be 
  compatible with previous releases.
 
 Heh, sure, we can do that too and it doesn't seem like anybody would
 object.  I will patch the documentation so that that the disabled
 value is zero, and still allow -1.  That way it doesn't seem like there
 should be any objection.

Patch attached.

-- 
Alvaro Herrera  http://www.amazon.com/gp/registry/5ZYLFMCVHXC
Saca el libro que tu religión considere como el indicado para encontrar la
oración que traiga paz a tu alma. Luego rebootea el computador
y ve si funciona (Carlos Duclós)
Index: doc/src/sgml/catalogs.sgml
===
RCS file: /home/alvherre/Code/cvs/pgsql/doc/src/sgml/catalogs.sgml,v
retrieving revision 2.153
diff -c -p -r2.153 catalogs.sgml
*** doc/src/sgml/catalogs.sgml	5 Jun 2007 21:31:03 -	2.153
--- doc/src/sgml/catalogs.sgml	8 Jun 2007 14:40:10 -
***
*** 1339,1347 
 be used for this particular value.  Observe that the
 structfieldvac_cost_delay/ variable inherits its default value from the
 xref linkend=guc-autovacuum-vacuum-cost-delay configuration parameter,
!or from varnamevacuum_cost_delay/ if the former is set to a negative
!value.  The same applies to structfieldvac_cost_limit/.
!Also, autovacuum will ignore attempts to set a per-table
 structfieldfreeze_max_age/ larger than the system-wide setting (it can only be set
 smaller), and the structfieldfreeze_min_age value/ will be limited to half the
 system-wide xref linkend=guc-autovacuum-freeze-max-age setting.
--- 1339,1350 
 be used for this particular value.  Observe that the
 structfieldvac_cost_delay/ variable inherits its default value from the
 xref linkend=guc-autovacuum-vacuum-cost-delay configuration parameter,
!or from xref linkend=guc-vacuum-cost-delay if the former is set to a negative
!value.  structfieldvac_cost_limit/ is an exception to this rule, because
!the value literal0/ is used to indicate that the
!xref linkend=guc-autovacuum-vacuum-cost-limit configuration parameter
!should be used, or xref linkend=guc-vacuum-cost-limit if the former is set to a
!zero or negative value.  Note that autovacuum will ignore attempts to set a per-table
 structfieldfreeze_max_age/ larger than the system-wide setting (it can only be set
 smaller), and the structfieldfreeze_min_age value/ will be limited to half the
 system-wide xref linkend=guc-autovacuum-freeze-max-age setting.
Index: doc/src/sgml/config.sgml
===
RCS file: /home/alvherre/Code/cvs/pgsql/doc/src/sgml/config.sgml,v
retrieving revision 1.126
diff -c -p -r1.126 config.sgml
*** doc/src/sgml/config.sgml	7 Jun 2007 19:19:56 -	1.126
--- doc/src/sgml/config.sgml	8 Jun 2007 14:15:33 -
*** SELECT * FROM parent WHERE key = 2400;
*** 3356,3362 
listitem
 para
  Specifies the cost limit value that will be used in automatic
! commandVACUUM/ operations.  If literal-1/ is specified (which is the
  default), the regular
  xref linkend=guc-vacuum-cost-limit value will be used.  Note that
  the value is distributed proportionally among the running autovacuum
--- 3356,3362 
listitem
 para
  Specifies the cost limit value that will be used in automatic
! commandVACUUM/ operations.  If literal0/ is specified (which is the
  default), the regular
  xref linkend=guc-vacuum-cost-limit value will be used.  Note that
  the value is distributed proportionally among the running autovacuum
Index: src/backend/postmaster/autovacuum.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/postmaster/autovacuum.c,v
retrieving revision 1.47
diff -c -p -r1.47 autovacuum.c
*** src/backend/postmaster/autovacuum.c	30 May 2007 20:11:57 -	1.47
--- src/backend/postmaster/autovacuum.c	8 Jun 2007 14:23:35 -
*** static void
*** 1548,1554 
  autovac_balance_cost(void)
  {
  	WorkerInfo	worker;
! 	int vac_cost_limit = (autovacuum_vac_cost_limit = 0 ?
    autovacuum_vac_cost_limit : VacuumCostLimit);
  	int vac_cost_delay = (autovacuum_vac_cost_delay = 0 ?
    autovacuum_vac_cost_delay : VacuumCostDelay);
--- 1548,1554 
  autovac_balance_cost(void)
  {
  	WorkerInfo	worker;
! 	int vac_cost_limit = (autovacuum_vac_cost_limit  0 ?
    autovacuum_vac_cost_limit : VacuumCostLimit);
  	int vac_cost_delay = (autovacuum_vac_cost_delay = 0 ?
    autovacuum_vac_cost_delay : VacuumCostDelay);
*** table_recheck_autovac(Oid relid)
*** 2143,2151 
  		 */
  		if (avForm != NULL)
  		{
! 			vac_cost_limit = 

Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit

2007-06-08 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes:
 Matthew T. O'Connor wrote:
 Can you make 0 and -1 both valid disabled values?  That way it will be 
 compatible with previous releases.
 
 Heh, sure, we can do that too and it doesn't seem like anybody would
 object.  I will patch the documentation so that that the disabled
 value is zero, and still allow -1.  That way it doesn't seem like there
 should be any objection.

 Patch attached.

It seems like documenting vac_cost_limit as being different from the
others will just create perceived complexity/confusion, with no real
benefit.  I'd suggest leaving the documentation and the default value
alone, and applying just the part of the patch that causes 0 to be
silently treated as if it were -1.

A comment at the spot where this is done would be a good idea, but
I don't think we need to say anything in the SGML docs.

regards, tom lane

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit

2007-06-07 Thread Alvaro Herrera
Matthew T. O'Connor wrote:
 Alvaro Herrera wrote:
 Tom Lane wrote:
 Alvaro Herrera [EMAIL PROTECTED] writes:
 But this is misleading (started postmaster with good value, then edited
 postgresql.conf and entered -2):
 17903 LOG:  received SIGHUP, reloading configuration files
 17903 LOG:  -2 is outside the valid range for parameter 
 autovacuum_vacuum_cost_limit (-1 .. 1000)
 Note how it still says the range is -1 .. 1000.
 Can we redefine things to make zero be the disabled value, thus
 keeping the range of valid values contiguous?
 
 That would be another solution ... though it would be different from the
 valid value for autovacuum_vacuum_cost_delay (on which 0 is a valid
 value).  Also it would be a different value from previous versions.
 
 I don't think either of these is a showstopper, so let's go for that if
 nobody objects.
 
 Can you make 0 and -1 both valid disabled values?  That way it will be 
 compatible with previous releases.

Heh, sure, we can do that too and it doesn't seem like anybody would
object.  I will patch the documentation so that that the disabled
value is zero, and still allow -1.  That way it doesn't seem like there
should be any objection.

-- 
Alvaro Herrera http://www.flickr.com/photos/alvherre/
Escucha y olvidarás; ve y recordarás; haz y entenderás (Confucio)

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] [PATCHES] xml2 contrib patch supporting default XML namespaces

2007-06-01 Thread Bruce Momjian

XML is now more stabilized in the backend than when the patch appeared,
and it doesn't make sense to add features to a /contrib interface that
is to be used only for backward compatability.

Patch rejected.  You can put the patch on pgfoundry if you are worried
others might need this functionality.

---

Mike Rylander wrote:
 On 3/6/07, Mike Rylander [EMAIL PROTECTED] wrote:
  On 3/6/07, Peter Eisentraut [EMAIL PROTECTED] wrote:
   Mike Rylander wrote:
The patch adds support for default XML namespaces in xml2 by providing
a mechanism for supplying a prefix to a named namespace URI.
  
   How does it support multiple namespaces in one document?
 
  It supports one default (unprefixed) namespace URI per document, which
  ISTM is the overwhelmingly common case (and the itch that I must
  scratch).
 
 I think there is some confusion about what the current xml2 contrib
 module supports and what my patch adds.  The current code, as it
 stands today, supports multiple namespaces just fine.  The only
 requirement is that each namespace have a prefix, or else one is
 forced to use the local-name() construct with every single node for
 those nodes in unprefixed (default) namespaces.  This patch simply
 adds support for registering a prefix for an unprefixed namespace,
 which is an extremely common case in XML and causes the use of overly
 verbose contortions when designing XPath expressions.  To illustrate
 this, xml2 currently supports all of these statements:
 
 SELECT xpath_nodeset('xyfoo/y/x','/x/y');
 SELECT xpath_nodeset('xa:y xmlns:a=uri:for:afoo/a:y/x','/x/a:y');
 SELECT xpath_nodeset('b:x xmlns:b=uri:for:ba:y
 xmlns:a=uri:for:afoo/a:y/b:x','/b:x/a:y');
 
 All number and manner of /prefixed/ namespaces work fine today.
 However, in order to match an element or attribute with an unprefixed
 namespace, the xpath becomes a study in overly verbose, human error
 inducing repetition.  For instance, consider the extremely common case
 of an xhtml document that does not use a prefix for the xhtml
 namespace.  Using the xml2 contrib module as it stands today, without
 my patch, using XPath to get the title of the document might look
 something like this:
 
 /*[local-name()=html]/*[local-name()=head]/*[local-name()=title]
 
 Now just imagine the XPath needed to get a portion of the body in a
 nested div based on the existence of some other node ... the logic
 gets lost in the noise simply because of the overhead of
 namespace-qualifying the elements.
 
 Namespaces were introduced in XML to address verbosity issues (among
 other things), but as XPath was designed primarily as a language for
 use inside XSLT (where namespace support is fully integrated) it
 didn't get the treatment needed to handle unprefixed namespaces.  To
 address /that/ issue, my patch allows the registration of a supplied
 prefix for a supplied URI, which solves the common default namespace
 problem in a completely backward compatible way.  The above example
 XPath can now become:
 
 /x:html/x:head/x:title
 
 simply by supplying 2 more arguments to the _ns version of any of the
 xpath_ functions available in xml2.  I challenge anyone to claim that
 the [local-name()=foo] variant is easier to read and less error prone
 than the second, namespace-prefixed variant.  They are exactly
 equivalent, but the second (quite obviously) is Better(tm).
 
 I understand that XML support is planned and at least partially
 implemented for 8.3, but many production instances will be unable (or,
 in fact, unwilling) to upgrade to 8.3 for quite some time.  Because
 this patch is completely backward compatible it can (theoretically) be
 included in future 8.1 and 8.2 releases, and for those of us that need
 more full XML support in the short term the upgrade of a contrib
 module is probably a very viable option -- it is for me, anyway.
 
 So, to sum up, please let me know what I can do to increase the
 chances of getting this patch included.  Alternatively, if my patch is
 being vetoed, please let me know that too so that I can create a local
 maintenance plan for this.
 
 Thanks in advance.  I've attached the patch again for reference.
 
 -- 
 Mike Rylander
 [EMAIL PROTECTED]
 GPLS -- PINES Development
 Database Developer
 http://open-ils.org

[ Attachment, skipping... ]

 
 ---(end of broadcast)---
 TIP 4: Have you searched our list archives?
 
http://archives.postgresql.org

-- 
  Bruce Momjian  [EMAIL PROTECTED]  http://momjian.us
  EnterpriseDB   http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-12 Thread Heikki Linnakangas

Tom Lane wrote:

Heikki Linnakangas [EMAIL PROTECTED] writes:
My first thought is that the cycle_ctr just adds extra complexity. The 
canceled-flag really is the key in Takahiro-san's patch, so we don't 
need the cycle_ctr anymore.


We don't have to have it in the sense of the code not working without it,
but it probably pays for itself by eliminating useless fsyncs.  The
overhead for it in my proposed implementation is darn near zero in the
non-error case.  Also, Takahiro-san mentioned at one point that he was
concerned to avoid useless fsyncs because of some property of the LDC
patch --- I wasn't too clear on what, but maybe he can explain.


Ok. Perhaps we should not use the canceled-flag but just remove the 
entry from pendingOpsTable like we used to when mdsync_in_progress isn't 
set. We might otherwise accumulate a lot of canceled entries in the hash 
table if checkpoint interval is long and relations are created and 
dropped as part of normal operation.


I think there's one little bug in the patch:

1. AbsorbFsyncRequests is called. A FORGET message is received, and an 
entry in the hash table is marked as canceled
2. Another relation with the same relfilenode is created. This can 
happen after OID wrap-around
3. RememberFsyncRequest is called for the new relation. The old entry is 
still in the hash table, marked with the canceled-flag, so it's not touched.


The fsync request for the new relation is masked by the old canceled 
entry. The trivial fix is to always clear the flag on step 3:


--- md.c2007-04-11 08:18:08.0 +0100
+++ md.c.new2007-04-12 09:21:00.0 +0100
@@ -1161,9 +1161,9 @@
 found);
if (!found) /* new entry, 
so initialize it */

{
-   entry-canceled = false;
entry-cycle_ctr = mdsync_cycle_ctr;
}
+   entry-canceled = false;
/*
 * NB: it's intentional that we don't change cycle_ctr 
if the entry
 * already exists.  The fsync request must be treated 
as old, even



--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-12 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes:
 I believe Itagaki-san's motivation for tackling this in the LDC patch 
 was the fact that it can fsync the same file many times, and in the 
 worst case go to an endless loop, and adding delays inside the loop 
 makes it much more likely. After that is fixed, I doubt any of the 
 optimizations of trying to avoid extra fsyncs make any difference in 
 real applications, and we should just keep it simple, especially if we 
 back-patch it.

I looked at the dynahash code and noticed that new entries are attached
to the *end* of their hashtable chain.  While this maybe should be
changed to link them at the front, the implication at the moment is that
without a cycle counter it would still be possible to loop indefinitely
because we'd continue to revisit the same file(s) after removing their
hashtable entries.  I think you'd need a constant stream of requests for
more than one file falling into the same hash chain, but it certainly
seems like a potential risk.  I'd prefer a solution that adheres to the
dynahash API's statement that it's unspecified whether newly-added
entries will be visited by hash_seq_search, and will in fact not loop
even if they always are visited.

 That said, I'm getting tired of this piece of code :).

Me too.

regards, tom lane

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-11 Thread Heikki Linnakangas

Tom Lane wrote:

I wrote:

Actually, on second look I think the key idea here is Takahiro-san's
introduction of a cancellation flag in the hashtable entries, to
replace the cases where AbsorbFsyncRequests can try to delete entries.
What that means is mdsync() doesn't need an outer retry loop at all:


I fooled around with this idea and came up with the attached patch.
It seems to do what's intended but could do with more eyeballs and
testing before committing.  Comments please?


I'm traveling today, but I'll take a closer look at it tomorrow morning. 
My first thought is that the cycle_ctr just adds extra complexity. The 
canceled-flag really is the key in Takahiro-san's patch, so we don't 
need the cycle_ctr anymore.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-11 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes:
 My first thought is that the cycle_ctr just adds extra complexity. The 
 canceled-flag really is the key in Takahiro-san's patch, so we don't 
 need the cycle_ctr anymore.

We don't have to have it in the sense of the code not working without it,
but it probably pays for itself by eliminating useless fsyncs.  The
overhead for it in my proposed implementation is darn near zero in the
non-error case.  Also, Takahiro-san mentioned at one point that he was
concerned to avoid useless fsyncs because of some property of the LDC
patch --- I wasn't too clear on what, but maybe he can explain.

regards, tom lane

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-10 Thread Tom Lane
I wrote:
 This patch looks fairly sane to me; I have a few small gripes about
 coding style but that can be fixed while applying.  Heikki, you were
 concerned about the cycle-ID idea; do you have any objection to this
 patch?

Actually, on second look I think the key idea here is Takahiro-san's
introduction of a cancellation flag in the hashtable entries, to
replace the cases where AbsorbFsyncRequests can try to delete entries.

What that means is mdsync() doesn't need an outer retry loop at all:
the periodic AbsorbFsyncRequests calls are not a hazard, and retry of
FileSync failures can be handled as an inner loop on the single failing
table entry.  (We can make the failure counter a local variable, too,
instead of needing space in every hashtable entry.)

And with that change, it's no longer possible for an incoming stream
of fsync requests to keep mdsync from terminating.  It might fsync
more than it really needs to, but it won't repeat itself, and it must
reach the end of the hashtable eventually.  So we don't actually need
the cycle counter at all.

It might be worth having the cycle counter anyway just to avoid doing
useless fsync work.  I'm not sure about this.  If we have a cycle
counter of say 32 bits, then it's theoretically possible for an fsync
to fail 2^32 consecutive times and then be skipped on the next try,
allowing a checkpoint to succeed that should not have.  We can fix that
with a few more lines of logic to detect a wrapped-around value, but is
it worth the trouble?

regards, tom lane

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-10 Thread Tom Lane
I wrote:
 Actually, on second look I think the key idea here is Takahiro-san's
 introduction of a cancellation flag in the hashtable entries, to
 replace the cases where AbsorbFsyncRequests can try to delete entries.
 What that means is mdsync() doesn't need an outer retry loop at all:

I fooled around with this idea and came up with the attached patch.
It seems to do what's intended but could do with more eyeballs and
testing before committing.  Comments please?

(Note: I ignored my own advice not to reindent.  Sorry ...)

regards, tom lane

*** src/backend/storage/smgr/md.c.orig  Wed Jan 17 11:25:01 2007
--- src/backend/storage/smgr/md.c   Tue Apr 10 16:38:27 2007
***
*** 122,135 
BlockNumber segno;  /* which segment */
  } PendingOperationTag;
  
  typedef struct
  {
PendingOperationTag tag;/* hash table key (must be first!) */
!   int failures;   /* number of failed 
attempts to fsync */
  } PendingOperationEntry;
  
  static HTAB *pendingOpsTable = NULL;
  
  
  typedef enum  /* behavior for mdopen  
_mdfd_getseg */
  {
--- 122,140 
BlockNumber segno;  /* which segment */
  } PendingOperationTag;
  
+ typedef uint16 CycleCtr;  /* can be any convenient integer size */
+ 
  typedef struct
  {
PendingOperationTag tag;/* hash table key (must be first!) */
!   boolcanceled;   /* T = request canceled, not 
yet removed */
!   CycleCtrcycle_ctr;  /* mdsync_cycle_ctr when 
request was made */
  } PendingOperationEntry;
  
  static HTAB *pendingOpsTable = NULL;
  
+ static CycleCtr mdsync_cycle_ctr = 0;
+ 
  
  typedef enum  /* behavior for mdopen  
_mdfd_getseg */
  {
***
*** 856,926 
  
  /*
   *mdsync() -- Sync previous writes to stable storage.
-  *
-  * This is only called during checkpoints, and checkpoints should only
-  * occur in processes that have created a pendingOpsTable.
   */
  void
  mdsync(void)
  {
!   boolneed_retry;
  
if (!pendingOpsTable)
elog(ERROR, cannot sync without a pendingOpsTable);
  
/*
!* The fsync table could contain requests to fsync relations that have
!* been deleted (unlinked) by the time we get to them.  Rather than
!* just hoping an ENOENT (or EACCES on Windows) error can be ignored,
!* what we will do is retry the whole process after absorbing fsync
!* request messages again.  Since mdunlink() queues a revoke message
!* before actually unlinking, the fsync request is guaranteed to be gone
!* the second time if it really was this case.  DROP DATABASE likewise
!* has to tell us to forget fsync requests before it starts deletions.
 */
!   do {
!   HASH_SEQ_STATUS hstat;
!   PendingOperationEntry *entry;
!   int absorb_counter;
  
!   need_retry = false;
  
/*
!* If we are in the bgwriter, the sync had better include all 
fsync
!* requests that were queued by backends before the checkpoint 
REDO
!* point was determined. We go that a little better by 
accepting all
!* requests queued up to the point where we start fsync'ing.
 */
!   AbsorbFsyncRequests();
  
!   absorb_counter = FSYNCS_PER_ABSORB;
!   hash_seq_init(hstat, pendingOpsTable);
!   while ((entry = (PendingOperationEntry *) 
hash_seq_search(hstat)) != NULL)
{
/*
!* If fsync is off then we don't have to bother opening 
the file
!* at all.  (We delay checking until this point so that 
changing
!* fsync on the fly behaves sensibly.)
 */
!   if (enableFsync)
{
SMgrRelation reln;
MdfdVec*seg;
  
/*
-* If in bgwriter, we want to absorb pending 
requests every so
-* often to prevent overflow of the fsync 
request queue.  This
-* could result in deleting the current entry 
out from under
-* our hashtable scan, so the procedure is to 
fall out of the
-* scan and start over from the top of the 
function.
-*/
-   if (--absorb_counter = 0)
-   {
-   need_retry = true;
-   break;
-

Re: [HACKERS] [PATCHES] Fix mdsync never-ending loop problem

2007-04-10 Thread Tom Lane
I wrote:
 I fooled around with this idea and came up with the attached patch.
 It seems to do what's intended but could do with more eyeballs and
 testing before committing.  Comments please?

Earlier I said that I didn't want to back-patch this change, but on
looking at the CVS history I'm reconsidering.  The performance problem
originates from the decision some time ago to do an AbsorbFsyncRequests
every so often during the mdsync loop; without that, and assuming no
actual failures, there isn't any absorption of new requests before
mdsync can complete.  Originally that code only existed in 8.2.x, but
very recently we back-patched it into 8.1.x as part of fixing the
file-deletion-on-Windows problem.  This means that 8.1.x users could
see a performance degradation upon updating to 8.1.8 from prior
subreleases, which wouldn't make them happy.

So I'm now thinking we ought to back-patch into 8.2.x and 8.1.x,
but of course that makes it even more urgent that we test the patch
thoroughly.

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] [PATCHES] Optimized pgbench for 8.3

2007-04-07 Thread Pavan Deolasee

On 4/6/07, Tatsuo Ishii [EMAIL PROTECTED] wrote:



BTW, is anybody working on enabling the fill factor to the tables used
by pgbench? 8.3 will introduce HOT, and I think adding the feature
will make it easier to test HOT.



Please see if the attached patch looks good. It adds a new -F option
which can be used to set fillfactor for tellers, accounts and branches
tables. Default is 100 and anything between 10 and 100 is acceptable.

Thanks,
Pavan

--

EnterpriseDB http://www.enterprisedb.com


pgbench_fillfactor.patch
Description: Binary data

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] [PATCHES] Optimized pgbench for 8.3

2007-04-07 Thread Tatsuo Ishii
Patch committed. Thanks.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

 On 4/6/07, Tatsuo Ishii [EMAIL PROTECTED] wrote:
 
 
  BTW, is anybody working on enabling the fill factor to the tables used
  by pgbench? 8.3 will introduce HOT, and I think adding the feature
  will make it easier to test HOT.
 
 
 Please see if the attached patch looks good. It adds a new -F option
 which can be used to set fillfactor for tellers, accounts and branches
 tables. Default is 100 and anything between 10 and 100 is acceptable.
 
 Thanks,
 Pavan
 
 -- 
 
 EnterpriseDB http://www.enterprisedb.com

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] [PATCHES] xml2 contrib patch supporting default XML namespaces

2007-03-22 Thread Bruce Momjian

Your patch has been added to the PostgreSQL unapplied patches list at:

http://momjian.postgresql.org/cgi-bin/pgpatches

It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.

---


Mike Rylander wrote:
 On 3/6/07, Mike Rylander [EMAIL PROTECTED] wrote:
  On 3/6/07, Peter Eisentraut [EMAIL PROTECTED] wrote:
   Mike Rylander wrote:
The patch adds support for default XML namespaces in xml2 by providing
a mechanism for supplying a prefix to a named namespace URI.
  
   How does it support multiple namespaces in one document?
 
  It supports one default (unprefixed) namespace URI per document, which
  ISTM is the overwhelmingly common case (and the itch that I must
  scratch).
 
 I think there is some confusion about what the current xml2 contrib
 module supports and what my patch adds.  The current code, as it
 stands today, supports multiple namespaces just fine.  The only
 requirement is that each namespace have a prefix, or else one is
 forced to use the local-name() construct with every single node for
 those nodes in unprefixed (default) namespaces.  This patch simply
 adds support for registering a prefix for an unprefixed namespace,
 which is an extremely common case in XML and causes the use of overly
 verbose contortions when designing XPath expressions.  To illustrate
 this, xml2 currently supports all of these statements:
 
 SELECT xpath_nodeset('xyfoo/y/x','/x/y');
 SELECT xpath_nodeset('xa:y xmlns:a=uri:for:afoo/a:y/x','/x/a:y');
 SELECT xpath_nodeset('b:x xmlns:b=uri:for:ba:y
 xmlns:a=uri:for:afoo/a:y/b:x','/b:x/a:y');
 
 All number and manner of /prefixed/ namespaces work fine today.
 However, in order to match an element or attribute with an unprefixed
 namespace, the xpath becomes a study in overly verbose, human error
 inducing repetition.  For instance, consider the extremely common case
 of an xhtml document that does not use a prefix for the xhtml
 namespace.  Using the xml2 contrib module as it stands today, without
 my patch, using XPath to get the title of the document might look
 something like this:
 
 /*[local-name()=html]/*[local-name()=head]/*[local-name()=title]
 
 Now just imagine the XPath needed to get a portion of the body in a
 nested div based on the existence of some other node ... the logic
 gets lost in the noise simply because of the overhead of
 namespace-qualifying the elements.
 
 Namespaces were introduced in XML to address verbosity issues (among
 other things), but as XPath was designed primarily as a language for
 use inside XSLT (where namespace support is fully integrated) it
 didn't get the treatment needed to handle unprefixed namespaces.  To
 address /that/ issue, my patch allows the registration of a supplied
 prefix for a supplied URI, which solves the common default namespace
 problem in a completely backward compatible way.  The above example
 XPath can now become:
 
 /x:html/x:head/x:title
 
 simply by supplying 2 more arguments to the _ns version of any of the
 xpath_ functions available in xml2.  I challenge anyone to claim that
 the [local-name()=foo] variant is easier to read and less error prone
 than the second, namespace-prefixed variant.  They are exactly
 equivalent, but the second (quite obviously) is Better(tm).
 
 I understand that XML support is planned and at least partially
 implemented for 8.3, but many production instances will be unable (or,
 in fact, unwilling) to upgrade to 8.3 for quite some time.  Because
 this patch is completely backward compatible it can (theoretically) be
 included in future 8.1 and 8.2 releases, and for those of us that need
 more full XML support in the short term the upgrade of a contrib
 module is probably a very viable option -- it is for me, anyway.
 
 So, to sum up, please let me know what I can do to increase the
 chances of getting this patch included.  Alternatively, if my patch is
 being vetoed, please let me know that too so that I can create a local
 maintenance plan for this.
 
 Thanks in advance.  I've attached the patch again for reference.
 
 -- 
 Mike Rylander
 [EMAIL PROTECTED]
 GPLS -- PINES Development
 Database Developer
 http://open-ils.org

[ Attachment, skipping... ]

 
 ---(end of broadcast)---
 TIP 4: Have you searched our list archives?
 
http://archives.postgresql.org

-- 
  Bruce Momjian  [EMAIL PROTECTED]  http://momjian.us
  EnterpriseDB   http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] [PATCHES] xpath_array with namespaces support

2007-03-22 Thread Bruce Momjian

Applying newest version of this patch now;  still needs documentation.

---

Nikolay Samokhvalov wrote:
 On 3/5/07, Nikolay Samokhvalov [EMAIL PROTECTED] wrote:
  On 3/4/07, Nikolay Samokhvalov [EMAIL PROTECTED] wrote:
   I'll fix these issues and extend the patch with resgression tests and
   docs for xpath_array(). I'll resubmit it very soon.
 
  Here is a new version of the patch. I didn't change any part of docs yet.
  Since there were no objections I've changed the name of the function
  to xmlpath().
 
 Updated version of the patch contains bugfix: there were a problem
 with path queries that pointed to elements (cases when a set of
 document parts that correspond to subtrees should be returned).
 Example is (included in regression test):
 
 xmltest=# SELECT xmlpath('//b', 'aone btwo/b three betc/b/a');
  xmlpath
 -
  {btwo/b,betc/b}
 (1 row)
 
 Waiting for more feedback, please check it.
 
 -- 
 Best regards,
 Nikolay

[ Attachment, skipping... ]

 
 ---(end of broadcast)---
 TIP 5: don't forget to increase your free space map settings

-- 
  Bruce Momjian  [EMAIL PROTECTED]  http://momjian.us
  EnterpriseDB   http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] [PATCHES] xml2 contrib patch supporting default XML namespaces

2007-03-22 Thread Peter Eisentraut
Bruce Momjian wrote:
 Your patch has been added to the PostgreSQL unapplied patches list
 at:

   http://momjian.postgresql.org/cgi-bin/pgpatches

 It will be applied as soon as one of the PostgreSQL committers
 reviews and approves it.

I was hoping that we're deprecating contrib/xml2, so I wouldn't add more 
features to it.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


  1   2   3   4   5   6   7   >