Re: [pgsql-www] [HACKERS] Upcoming PG re-releases

2005-12-01 Thread Richard Huxton
Robert Treat wrote: On Wed, 2005-11-30 at 13:33, Magnus Hagander wrote: Someone suggested earlier that we should drop the binaries for nonsupported versions completely from the ftp site. Thoughts on this? If not, they should at least go into OLD as well. But personally, I'm for dropping them

[HACKERS] generalizing the planner knobs

2005-12-01 Thread Neil Conway
There are currently some rather crude knobs for persuading the planner to favour certain kinds of query plans: the enable_XXX GUC variables. Several people have asked for a more flexible way to give hints to the planner. I'm not interested in implementing fully-general planner hints at the moment,

[HACKERS] Docs misspelling

2005-12-01 Thread Christopher Kings-Lynne
36.7.3.5. FOR (integer variant) In the 8.1 docs. Label has been spelt Labal. Chris ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] Docs misspelling

2005-12-01 Thread Neil Conway
On Thu, 2005-12-01 at 18:33 +0800, Christopher Kings-Lynne wrote: 36.7.3.5. FOR (integer variant) In the 8.1 docs. Label has been spelt Labal. Thanks, fixed in HEAD and REL8_1_STABLE. -Neil ---(end of broadcast)--- TIP 1: if posting/reading

Re: [pgsql-www] [HACKERS] Upcoming PG re-releases

2005-12-01 Thread Euler Taveira de Oliveira
--- Richard Huxton dev@archonet.com escreveu: If it's practical to keep them, I'd like to suggest doing so. If it's not practical, could we have a where_to_find_old_versions.txt file and open a project on sourceforge to keep them? What about an museum.postgresql.org to keep the old

Re: [HACKERS] Another way to reduce pg_subtrans lookup overhead

2005-12-01 Thread Alvaro Herrera
Tom Lane wrote: I mentioned yesterday that I'm looking at the problem of excessive accesses to pg_subtrans when there is an old open transaction: http://archives.postgresql.org/pgsql-hackers/2005-11/msg01547.php I thought of a different approach to it, which is to make snapshot checking

[HACKERS] Vertical Partitioning with TOAST

2005-12-01 Thread Junji TERAMOTO
Hi all, I wrote a experimental patch for a vertical partitioning function. I decided to use the code of TOAST to create the function easily. In a word, the row that the user specified is forcedly driven out with TOAST. The performance gain of 10% was seen by driving out c_data of the customer

Re: [pgsql-www] [HACKERS] Upcoming PG re-releases

2005-12-01 Thread Peter Eisentraut
Am Donnerstag, 1. Dezember 2005 11:35 schrieb Euler Taveira de Oliveira: What about an museum.postgresql.org to keep the old releases? That gave me a good laugh, but there is something to be said about moving all no longer supported releases (according to the criteria that are being discussed)

Re: [pgsql-www] [HACKERS] Upcoming PG re-releases

2005-12-01 Thread Csaba Nagy
Maybe mausoleum would be even better name :-D Cheers, Csaba. On Thu, 2005-12-01 at 11:35, Euler Taveira de Oliveira wrote: --- Richard Huxton dev@archonet.com escreveu: If it's practical to keep them, I'd like to suggest doing so. If it's not practical, could we have a

Re: [pgsql-www] [HACKERS] Upcoming PG re-releases

2005-12-01 Thread Richard Huxton
Csaba Nagy wrote: Maybe mausoleum would be even better name :-D Come on people, it's clearly: elephants-graveyard.postgresl.org -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an

Re: [HACKERS] [PATCHES] A couple of proposed pgbench changes

2005-12-01 Thread Tom Lane
Tatsuo Ishii [EMAIL PROTECTED] writes: ... I wonder if it would help for pgbench to fork off multiple sub-processes and have each sub-process tend just one backend. I'm not sure multiple sub-processes version of pgbench shows superior performance than current implementation because of

Re: [HACKERS] Another way to reduce pg_subtrans lookup overhead

2005-12-01 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: I thought of a different approach to it, which is to make snapshot checking take a hint from TransactionIdIsInProgress: use the subxid caches from the PG_PROC array. Yeah, I had thought about using the XidCache while checking your

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Jonah H. Harris
Hey Neil, In the last couple weeks I too have been thinking about planner hints. Assuming I have read your post correctly, the issue I see with this idea is that, in most cases, there won't be much of a difference between adding an arbitrary cost value to each type of node and disabling it

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: ... ISTM that a simple improvement to what we have now would allow for a wider range of planner hints with only minor changes: we could replace the enable_XXX variables with a set of variables that would add an arbitrary constant to the estimated cost of

Re: [pgsql-www] [HACKERS] Upcoming PG re-releases

2005-12-01 Thread Marc G. Fournier
On Thu, 1 Dec 2005, Peter Eisentraut wrote: Am Donnerstag, 1. Dezember 2005 11:35 schrieb Euler Taveira de Oliveira: What about an museum.postgresql.org to keep the old releases? That gave me a good laugh, but there is something to be said about moving all no longer supported releases

Re: [pgsql-www] [HACKERS] Upcoming PG re-releases

2005-12-01 Thread Dave Page
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marc G. Fournier Sent: 01 December 2005 17:01 To: Peter Eisentraut Cc: pgsql-hackers@postgresql.org; Euler Taveira de Oliveira; Richard Huxton; Robert Treat; Magnus Hagander; Marc G. Fournier;

Re: [HACKERS] Shared locking in slru.c

2005-12-01 Thread Kenneth Marshall
On Wed, Nov 30, 2005 at 03:23:55PM -0500, Tom Lane wrote: Kenneth Marshall [EMAIL PROTECTED] writes: ... In pseudo-code, the operations to read the control information are: WriteControl: 1. Set latch. 2. Update control information 3. Increment latch version number. 4. Unset latch.

Re: [HACKERS] [ADMIN] ERROR: could not read block

2005-12-01 Thread Kevin Grittner
Due to its size, in the Windows environment we can't dump this database in any format except plain text, so the zlib issues don't apply here. -Kevin Qingqing Zhou [EMAIL PROTECTED] By they way, they found that they were getting this on a pg_dump, too. We will test both failure cases. If

Re: [HACKERS] [ADMIN] ERROR: could not read block

2005-12-01 Thread Kevin Grittner
[Apologies for the delayed response; fighting through a backlog.] I checked with out DBAs, and they are willing to test it. By they way, they found that they were getting this on a pg_dump, too. We will test both failure cases. If the test goes OK, we would be happy to leave it in production

Re: [HACKERS] Shared locking in slru.c

2005-12-01 Thread Kenneth Marshall
On Wed, Nov 30, 2005 at 01:53:13PM -0500, Tom Lane wrote: I've been looking at various ways to resolve this, but one thing that seems promising is to hack slru.c to take the control lock in shared mode, not exclusive mode, for read-only accesses to pages that are already in memory. The vast

Re: [pgsql-www] [HACKERS] Upcoming PG re-releases

2005-12-01 Thread Andrew Dunstan
Dave Page wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marc G. Fournier Sent: 01 December 2005 17:01 To: Peter Eisentraut Cc: pgsql-hackers@postgresql.org; Euler Taveira de Oliveira; Richard Huxton; Robert Treat; Magnus Hagander;

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Qingqing Zhou
Neil Conway [EMAIL PROTECTED] wrote This would also be useful when diagnosing bad query plans: for example, setting enable_seqscan=false often causes the planner to disregard the use of *any* sequential scan, anywhere in the plan. The ability to slightly bump up the cost of particular

Re: [HACKERS] [PATCHES] A couple of proposed pgbench changes

2005-12-01 Thread Qingqing Zhou
Tom Lane [EMAIL PROTECTED] wrote I don't know yet why pgbench would be able to affect this, but it's repeatable. If anyone's interested in trying to duplicate it, I'll post my version of pgbench. I'd like to try to duplicate it here, Regards, Qingqing ---(end

Re: [HACKERS] Strange interval arithmetic

2005-12-01 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: Michael Fuhr [EMAIL PROTECTED] writes: I usually check both in my own code but I noticed several places where PostgreSQL doesn't, so I kept that style. I'll check both if that's preferred. I'd say not --- it's more code and it makes a possibly

Re: [HACKERS] Strange interval arithmetic

2005-12-01 Thread Greg Stark
Generally speaking looking at errno when you haven't received an error return from a libc function is asking for trouble. It could be leftover from any previous libc error. Oh, sorry, just reread the code snippet and see that isn't an issue. nevermind. -- greg

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Tom Lane
Jonah H. Harris [EMAIL PROTECTED] writes: In the last couple weeks I too have been thinking about planner hints. Assuming I have read your post correctly, the issue I see with this idea is that, in most cases, there won't be much of a difference between adding an arbitrary cost value to each

Re: [pgsql-www] [HACKERS] Upcoming PG re-releases

2005-12-01 Thread Magnus Hagander
That would be fairly trivial ... let me add it to the 'todo list' ... I take it that it would be safe to relegate the /pub/source/OLD stuff there too? Not so trivial to put behind a web interface or the download tracker though. Is it really necessary to have a separate archive

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Jonah H. Harris
Tom, Don't get me wrong, I agree with you completely. I would rather put effort into enhancing the planner than in developing work-arounds. In 99% of all cases the planner works correctly, but I know people who actually have to disable planning options (mergejoin) in production applications

Re: [HACKERS] [ADMIN] ERROR: could not read block

2005-12-01 Thread Tom Lane
Qingqing Zhou [EMAIL PROTECTED] writes: I come up with a patch to fix server-side problem. Applied. For windows, I set a one second waiting time - The code actually does one millisecond; I left it that way since it seems a reasonable value. regards, tom lane

Re: [HACKERS] Strange interval arithmetic

2005-12-01 Thread Greg Stark
Greg Stark [EMAIL PROTECTED] writes: Generally speaking looking at errno when you haven't received an error return from a libc function is asking for trouble. It could be leftover from any previous libc error. That's how you get programs saying things like strtol: No such file or

Re: [HACKERS] Strange interval arithmetic

2005-12-01 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: Ah, I take back my taking back of this. It's still dicey to be doing it this way -- even if you reset errno before calling the library function. See the rest of the thread. The I-believe-what-I-read-in-the-spec camp may take comfort from what it says in the

Re: [HACKERS] Strange interval arithmetic

2005-12-01 Thread Michael Fuhr
On Thu, Dec 01, 2005 at 03:31:41PM -0500, Greg Stark wrote: Greg Stark [EMAIL PROTECTED] writes: Generally speaking looking at errno when you haven't received an error return from a libc function is asking for trouble. It could be leftover from any previous libc error. That's how

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Greg Stark
Jonah H. Harris [EMAIL PROTECTED] writes: Tom, Don't get me wrong, I agree with you completely. I would rather put effort into enhancing the planner than in developing work-arounds. In 99% of all cases the planner works correctly, but I know people who actually have to disable planning

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is set to zero.

2005-12-01 Thread Bruce Momjian
Tom Lane wrote: [EMAIL PROTECTED] (Bruce Momjian) writes: Log Message: --- Add comments about why errno is set to zero. These comments seem a bit wrongheaded, since checking LONG_MIN/LONG_MAX is exactly not what we could do to detect an overflow error. Yea, I noticed the 0 was

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: On the other hand the type I would prefer to see are hints that feed directly into filling in information the planner lacks. This only requires that the user understand his own data and still lets the planner pick the best plan based on the provided

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is set to zero.

2005-12-01 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: Should I just change them all to: errno = 0; /* avoid checking result for failure */ No, that's still a completely inaccurate description of the reason for having the statement. or should I add a macro to c.h as: /* Sometimes we

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is set to zero.

2005-12-01 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Should I just change them all to: errno = 0; /* avoid checking result for failure */ No, that's still a completely inaccurate description of the reason for having the statement. or should I add a macro to c.h as:

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Pollard, Mike
Greg Stark [EMAIL PROTECTED] writes: On the other hand the type I would prefer to see are hints that feed directly into filling in information the planner lacks. This only requires that the user understand his own data and still lets the planner pick the best plan based on the provided

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is set to zero.

2005-12-01 Thread Alvaro Herrera
Bruce Momjian wrote: Tom Lane wrote: or should I add a macro to c.h as: /* Sometimes we need to clear errno so we can check errno * without having to check for a failure value from the function * call. */ #define CLEAR_ERRNO \\ do { \

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is set to zero.

2005-12-01 Thread Martijn van Oosterhout
On Thu, Dec 01, 2005 at 04:12:30PM -0500, Bruce Momjian wrote: Well, there seems to be enough confusion, even in this email list, that identifying _why_ errno is being cleared is a good idea. I modified it to: errno = 0; /* avoid having to check the result for failure */ I don't

Re: [HACKERS] SHOW ALL output too wide

2005-12-01 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Dennis Bjorklund wrote: My motivation is only for the good of postgresql. If the majority want something else then that's what should happen (of course). I'm just stating my view, nothing less and nothing more. I am

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is set

2005-12-01 Thread Bruce Momjian
Martijn van Oosterhout wrote: -- Start of PGP signed section. On Thu, Dec 01, 2005 at 04:12:30PM -0500, Bruce Momjian wrote: Well, there seems to be enough confusion, even in this email list, that identifying _why_ errno is being cleared is a good idea. I modified it to:

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is

2005-12-01 Thread Neil Conway
On Thu, 2005-12-01 at 16:38 -0500, Bruce Momjian wrote: Maybe it should be: errno = 0; /* Allow unconditional errno check */ I think any solution that involves adding more duplication at each strtol() callsite is not great (Don't Repeat Yourself). I'd still like to see this

Re: [HACKERS] Improving count(*)

2005-12-01 Thread Bruce Momjian
Add idea to TODO: * Allow data to be pulled directly from indexes Currently indexes do not have enough tuple visibility information to allow data to be pulled from the index without also accessing the heap. One way to allow this is to set a bit on

Re: [HACKERS] Fork-based version of pgbench

2005-12-01 Thread Tom Lane
Now that I've fixed the silly mistake in the fork-based version of pgbench, http://archives.postgresql.org/pgsql-patches/2005-12/msg00017.php I'm seeing it consistently outperform the CVS-tip version by about 5%. I get about 700 tps versus 670 tps; meanwhile top reports that idle CPU percentage

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is

2005-12-01 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: If people would like to see a detailed explanation of the interaction between strtol() and errno, a header comment to pg_strtol() seems a good place to put it. IMO that is better than copying and pasting a cryptic one-line comment to each and every

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is set to zero.

2005-12-01 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: I modified it to: errno = 0; /* avoid having to check the result for failure */ Just for the record, that's *still* wrong. It implies that if we tested (result == LONG_MAX errno == ERANGE), without zeroing errno beforehand, the code would

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is set to zero.

2005-12-01 Thread Tom Lane
Martijn van Oosterhout kleptog@svana.org writes: errno = 0; /* clear prior detected errors */ That one is at least a correct explanation of what the code is doing... regards, tom lane ---(end of broadcast)--- TIP

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Gregory Maxwell
On 12/1/05, Pollard, Mike [EMAIL PROTECTED] wrote: Optimizer hints were added because some databases just don't have a very smart optimizer. But you are much better served tracking down cases in which the optimizer makes a bad choice, and teaching the optimizer how to make a better one. That

[HACKERS] Reducing relation locking overhead

2005-12-01 Thread Tom Lane
In looking at the current pgbench results, I notice that one considerable contribution to LWLock contention is access to the heavyweight-lock manager. Almost all of that comes from taking relation-level locks, so we could cut down the contention if we could reduce the number of distinct locks

Re: [HACKERS] Reducing relation locking overhead

2005-12-01 Thread Christopher Kings-Lynne
4. The only reason we need to take relation-level locks on indexes at all is to make the world safe for REINDEX being done concurrently with read-only accesses to the table (that don't use the index being reindexed). If we went back to requiring exclusive lock for reindex we could forget all

Re: [HACKERS] [COMMITTERS] pgsql: Add comments about why errno is set

2005-12-01 Thread Bruce Momjian
OK, comments removed, and comment added to port/strtol.c. --- Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: I modified it to: errno = 0; /* avoid having to check the result for failure */

Re: [HACKERS] Windows installation notes

2005-12-01 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Thanks for responding to this and fixing what you could. I understand now that a lot of the things are Window-isms and thus not fixable. It's the default size for a Windows Installer installation program. We could make it larger, but then we'd

Re: [HACKERS] Reducing relation locking overhead

2005-12-01 Thread Stephen Frost
* Christopher Kings-Lynne ([EMAIL PROTECTED]) wrote: 4. The only reason we need to take relation-level locks on indexes at all is to make the world safe for REINDEX being done concurrently with read-only accesses to the table (that don't use the index being reindexed). If we went back to

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Pollard, Mike
Gregory Maxwell [EMAIL PROTECTED] wrote: The flipside there is that a good set of hinting options may increase the amount of detailed feedback we get from users on improvements needed in the optimizer. The current knobs are pretty blunt and don't do as much as I'd like when trying to track

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Neil Conway
On Thu, 2005-12-01 at 21:01 -0500, Gregory Maxwell wrote: If we'd really like to avoid people using the knobs to rig queries, how about making them only work with explain analyze, useful for debugging but not so useful for actual queries. That seems a pretty arbitrary limitation. I agree that

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: On Thu, 2005-12-01 at 21:01 -0500, Gregory Maxwell wrote: If we'd really like to avoid people using the knobs to rig queries, how about making them only work with explain analyze, useful for debugging but not so useful for actual queries. That seems a

[HACKERS] Weird Grant/Revoke/Usage behavior

2005-12-01 Thread Joshua D. Drake
Hello, The below seems incorrect. If I am in the schema the behavior seems correct. I can't see or select from the table. However if I am not in the schema I am able to see the table and its structure. The user jd is not a superuser. cleancontact=# revoke usage on schema financials from jd;

[HACKERS] Additional Grant/revoke problem

2005-12-01 Thread Joshua D. Drake
Hello, CREATE TABLE foo (id bigserial primary key, fname text); GRANT INSERT ON foo to jd; INSERT INTO foo(fname) VALUES ('Joshua'); This will fail, because I don't have permissions on the sequence foo_id_seq which is a dependency created by PostgreSQL. It seems that if bigserial is the

Re: [HACKERS] Fork-based version of pgbench

2005-12-01 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: It's not so much that I want to inflate the measurements, as that leaving 10% of the CPU on the table reduces pgbench's usefulness as a way of stress-testing the backend. I suspect the difference is the same thing you theorised made the difference before.

Re: [HACKERS] Reducing relation locking overhead

2005-12-01 Thread Greg Stark
Christopher Kings-Lynne [EMAIL PROTECTED] writes: Surely in the real world REINDEX is run so rarely compared to all those other operations it'd be a win... It's not a question of frequency. We're not talking about something like a 10% performance loss. You're talking about whether REINDEX is

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Trent Shipley
On Thursday 2005-12-01 19:01, Gregory Maxwell wrote: On 12/1/05, Pollard, Mike [EMAIL PROTECTED] wrote: Optimizer hints were added because some databases just don't have a very smart optimizer. But you are much better served tracking down cases in which the optimizer makes a bad choice,

Re: [HACKERS] generalizing the planner knobs

2005-12-01 Thread Greg Stark
Pollard, Mike [EMAIL PROTECTED] writes: Optimizer hints were added because some databases just don't have a very smart optimizer. But you are much better served tracking down cases in which the optimizer makes a bad choice, and teaching the optimizer how to make a better one. You more or

Re: [HACKERS] Fork-based version of pgbench

2005-12-01 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: It's not so much that I want to inflate the measurements, as that leaving 10% of the CPU on the table reduces pgbench's usefulness as a way of stress-testing the backend. ... In any case it seems like there would be

Re: [HACKERS] Reducing relation locking overhead

2005-12-01 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: It was a *major* new feature that many people were waiting for when Oracle finally implemented live CREATE INDEX and REINDEX. The ability to run create an index without blocking any operations on a table, even updates, was absolutely critical for 24x7

[HACKERS] Buildfarm: Bear, Branch 2?

2005-12-01 Thread Michael Glaesemann
Out of curiosity, what is this beast? http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=beardt=2005-11-13% 2012:01:08 Michael Glaesemann grzm myrealbox com ---(end of broadcast)--- TIP 4: Have you searched our list archives?