Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Michael Paesold
Matthew T. O'Connor schrieb: Tom Lane wrote: Andrew Hammond [EMAIL PROTECTED] writes: Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be better to say that setting the naptime really high is a Bad Idea. It seems

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Zeugswetter Andreas ADI SD
The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. Andreas ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-08 Thread Heikki Linnakangas
Greg Smith wrote: On Thu, 7 Jun 2007, Heikki Linnakangas wrote: So there's two extreme ways you can use LDC: 1. Finish the checkpoint as soon as possible, without disturbing other activity too much 2. Disturb other activity as little as possible, as long as the checkpoint finishes in a

Re: [HACKERS] little PITR annoyance

2007-06-08 Thread ohp
Hi Simon, On Thu, 7 Jun 2007, Simon Riggs wrote: Date: Thu, 07 Jun 2007 23:10:06 +0100 From: Simon Riggs [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: pgsql-hackers list pgsql-hackers@postgresql.org Subject: Re: [HACKERS] little PITR annoyance On Thu, 2007-06-07 at 17:01 +0200, [EMAIL

Re: [HACKERS] TOAST usage setting

2007-06-08 Thread Zeugswetter Andreas ADI SD
My next suggestion would be to leave EXTERN_TUPLES_PER_PAGE as is, but: Split data wider than a page into page sized chunks as long as they fill whole pages. Split the rest with EXTERN_TUPLES_PER_PAGE (4) as now. This would not waste more space than currently, but improve performance

Re: [HACKERS] Performance regression on CVS head

2007-06-08 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: I tried to repeat the DBT-2 runs with the oldestxmin refresh patch, but to my surprise the baseline run with CVS head, without the patch, behaved very differently than it did back in March. I rerun the a shorter 1h test with CVS

[HACKERS] COPYable logs status

2007-06-08 Thread Andrew Dunstan
[summarising discussion on -patches] The situation with this patch is that I now have it in a state where I think it could be applied, but there is one blocker, namely that we do not have a way of preventing the interleaving of log messages from different backends, which leads to garbled

[HACKERS] Index adviser or just [pg_]adviser

2007-06-08 Thread Gurjeet Singh
I am done with refactoring the Index adviser code to make use of the current planner hooks, and as part of this drive I was thinking of changing the name of the feature/plugin too. Since there's a possibility that the same architecture can be used by other types of adviser plugins

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Alvaro Herrera
Andrew Dunstan wrote: Unfortunately, there is no solution in sight for this problem, certainly not one which I think can be devised and implemented simply at this stage of the cycle. The solution we'd like to use, LWLocks, is not workable in his context. In consequence, I don't think we

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Alvaro Herrera
Zeugswetter Andreas ADI SD escribió: The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. This is what we have. The sleep time depends on the schedule of next vacuum for the closest database in time. If naptime is

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Alvaro Herrera
Martijn van Oosterhout wrote: On Fri, Jun 08, 2007 at 08:31:54AM -0400, Andrew Dunstan wrote: The situation with this patch is that I now have it in a state where I think it could be applied, but there is one blocker, namely that we do not have a way of preventing the interleaving of log

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Matthew O'Connor
Michael Paesold wrote: Matthew T. O'Connor schrieb: Do we need a configurable autovacuum naptime at all? I know I put it in the original contrib autovacuum because I had no idea what knobs might be needed. I can't see a good reason to ever have a naptime longer than the default 60 seconds,

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Martijn van Oosterhout
On Fri, Jun 08, 2007 at 08:31:54AM -0400, Andrew Dunstan wrote: The situation with this patch is that I now have it in a state where I think it could be applied, but there is one blocker, namely that we do not have a way of preventing the interleaving of log messages from different

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Martijn van Oosterhout wrote: The whole semantics of PIPEBUF should prevent garbling, as long as each write is a complete set of lines and no more than PIPEBUF bytes long. Have we determined the actual cause of the garbling? No, that's the main

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-08 Thread Andrew Sullivan
On Fri, Jun 08, 2007 at 09:50:49AM +0100, Heikki Linnakangas wrote: dynamics change. But we must also keep in mind that average DBA doesn't change any settings, and might not even be able or allowed to. That means the defaults should work reasonably well without tweaking the OS settings.

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-08 Thread Greg Smith
On Fri, 8 Jun 2007, Andrew Sullivan wrote: Do you mean change the OS settings or something else? (I'm not sure it's true in any case, because shared memory kernel settings have to be fiddled with in many instances, but I thought I'd ask for clarification.) In a situation where a hosting

Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit

2007-06-08 Thread Alvaro Herrera
Alvaro Herrera wrote: Matthew T. O'Connor wrote: Can you make 0 and -1 both valid disabled values? That way it will be compatible with previous releases. Heh, sure, we can do that too and it doesn't seem like anybody would object. I will patch the documentation so that that the

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-08 Thread Heikki Linnakangas
Andrew Sullivan wrote: On Fri, Jun 08, 2007 at 09:50:49AM +0100, Heikki Linnakangas wrote: dynamics change. But we must also keep in mind that average DBA doesn't change any settings, and might not even be able or allowed to. That means the defaults should work reasonably well without

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: The idea of one pipe per process is not really workable, because it would mean having as many pipes as backends which does not sound very good. But how about a mixed approach -- like have the all the backends share a pipe, controlled by an LWLock, and

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Andrew Dunstan
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Martijn van Oosterhout wrote: The whole semantics of PIPEBUF should prevent garbling, as long as each write is a complete set of lines and no more than PIPEBUF bytes long. Have we determined the actual cause of the garbling?

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Andrew Sullivan
On Fri, Jun 08, 2007 at 10:29:03AM -0400, Tom Lane wrote: The only solution that I can see is to invent some sort of simple protocol for the syslogger pipe. Perhaps having a look at the current IETF syslog discussion will be helpful in that case? (I know it's not directly relevant, but

Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit

2007-06-08 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Matthew T. O'Connor wrote: Can you make 0 and -1 both valid disabled values? That way it will be compatible with previous releases. Heh, sure, we can do that too and it doesn't seem like anybody would object. I will patch the documentation so that

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-08 Thread Andrew Sullivan
On Fri, Jun 08, 2007 at 10:33:50AM -0400, Greg Smith wrote: they'd take care of that as part of routine server setup. What wouldn't be reasonable is to expect them to tune obscure parts of the kernel just for your application. Well, I suppose it'd depend on what kind of hosting environment

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Alvaro Herrera
Zeugswetter Andreas ADI SD escribió: The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. This is what we have. The sleep time depends on the schedule of next vacuum for the closest database in time.

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Matthew T. O'Connor
Andrew Dunstan wrote: The situation with this patch is that I now have it in a state where I think it could be applied, but there is one blocker, namely that we do not have a way of preventing the interleaving of log messages from different backends, which leads to garbled logs. This is an

[HACKERS] Using the GPU

2007-06-08 Thread Billings, John
Does anyone think that PostgreSQL could benefit from using the video card as a parallel computing device? I'm working on a project using Nvidia's CUDA with an 8800 series video card to handle non-graphical algorithms. I'm curious if anyone thinks that this technology could be used to speed up

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Zeugswetter Andreas ADI SD
The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. This is what we have. The sleep time depends on the schedule of next vacuum for the closest database in time. If naptime is high, the sleep time will be

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-08 Thread Bruce Momjian
Andrew Sullivan wrote: On Fri, Jun 08, 2007 at 10:33:50AM -0400, Greg Smith wrote: they'd take care of that as part of routine server setup. What wouldn't be reasonable is to expect them to tune obscure parts of the kernel just for your application. Well, I suppose it'd depend on what

Re: [HACKERS] little PITR annoyance

2007-06-08 Thread Simon Riggs
Hi Olivier, On Fri, 2007-06-08 at 11:41 +0200, [EMAIL PROTECTED] wrote: The problem is not with gzip or scp, it is that postmaster refuses to start because wal FS is full. My questions was: why don't we start the archiving *BEFORE* postmaster to make room. The archiver is executed from the

Re: [HACKERS] Minor changes to Recovery related code

2007-06-08 Thread Simon Riggs
On Thu, 2007-06-07 at 21:53 +0100, Simon Riggs wrote: We have a number of problems surrounding pg_stop_backup/shutdown: I'll start coding up my proposals, in the absence of alternate views. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com

[HACKERS] Patch review process; Warning to patch submitters

2007-06-08 Thread Bruce Momjian
Based on the patches status page: http://developer.postgresql.org/index.php/Todo:PatchStatus#current_8.3_patch_status we are a few weeks away from completing all patch review and entering beta. If you have been asked to updated your patch by a patch reviewer, please do so in the next

Re: [HACKERS] Using the GPU

2007-06-08 Thread Vincent Janelle
Aren't most databases constrained by I/O? And postgresql by how fast your kernel can switch between processes under a concurrent load? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Billings, John Sent: Friday, June 08, 2007 10:55 AM To: pgsql-hackers@postgresql.org Subject:

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Tom Lane
Matthew T. O'Connor [EMAIL PROTECTED] writes: How about creating a log-writing-process? Postmaster could write to the log files directly until the log-writer is up and running, then all processes can send their log output through the log-writer. We *have* a log-writing process. The problem

Re: [HACKERS] COPYable logs status

2007-06-08 Thread Matthew T. O'Connor
Tom Lane wrote: Matthew T. O'Connor [EMAIL PROTECTED] writes: How about creating a log-writing-process? Postmaster could write to the log files directly until the log-writer is up and running, then all processes can send their log output through the log-writer. We *have* a log-writing

[HACKERS] Binary mode copy in from stdin

2007-06-08 Thread Dann Corbit
We have (among other things) and ODBC/OLEDB/JDBC/.NET driver for PostgreSQL and we want to optimize fast mode insert/select behavior. When we try to do a binary mode copy from standard input, we get an error message that we can't do it. How can we programmatically insert data using COPY

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Jim C. Nasby
On Fri, Jun 08, 2007 at 09:49:56AM -0400, Matthew O'Connor wrote: Michael Paesold wrote: Matthew T. O'Connor schrieb: Do we need a configurable autovacuum naptime at all? I know I put it in the original contrib autovacuum because I had no idea what knobs might be needed. I can't see a

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Jim C. Nasby
On Thu, Jun 07, 2007 at 12:13:09PM -0700, Andrew Hammond wrote: On 6/7/07, Jim C. Nasby [EMAIL PROTECTED] wrote: On Mon, Jun 04, 2007 at 11:04:26AM -0400, Alvaro Herrera wrote: The launcher is set up to wake up in autovacuum_naptime seconds at most. So if the user configures a ridiculuos

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Alvaro Herrera
Jim C. Nasby escribió: There *is* reason to allow setting the naptime smaller, though (or at least there was; perhaps Alvero's recent changes negate this need): clusters that have a large number of databases. I've worked with folks who are in a hosted environment and give each customer their

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Jim C. Nasby escribió: There *is* reason to allow setting the naptime smaller, though (or at least there was; perhaps Alvero's recent changes negate this need): clusters that have a large number of databases. I've worked with folks who are in a hosted environment and give

Re: [HACKERS] Patch review process; Warning to patch submitters

2007-06-08 Thread Alvaro Herrera
Simon Riggs wrote: On Fri, 2007-06-08 at 16:29 -0400, Bruce Momjian wrote: The bottom line is that the patch review process is hard enough, and we need the assistance of patch submitters to work with the patch reviewers in a timely manner. This includes me, so: Understood. There is

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Alvaro Herrera
Matthew T. O'Connor escribió: Ok, but I think the question posed is that in say a virtual hosting environment there might be say 1,000 databases in the cluster. Am I still going to have to wait a long time for my database to get vacuumed? I don't think this has changed much no? Depends

Re: [HACKERS] Patch review process; Warning to patch submitters

2007-06-08 Thread Simon Riggs
On Fri, 2007-06-08 at 16:29 -0400, Bruce Momjian wrote: The bottom line is that the patch review process is hard enough, and we need the assistance of patch submitters to work with the patch reviewers in a timely manner. This includes me, so: Understood. There is a patch missing from the

Re: [HACKERS] Using the GPU

2007-06-08 Thread Gregory Stark
Billings, John [EMAIL PROTECTED] writes: Does anyone think that PostgreSQL could benefit from using the video card as a parallel computing device? I'm working on a project using Nvidia's CUDA with an 8800 series video card to handle non-graphical algorithms. I'm curious if anyone thinks

Re: [HACKERS] Command tags in create/drop scripts

2007-06-08 Thread Jim C. Nasby
On Tue, Jun 05, 2007 at 05:52:39PM -, Andrew Hammond wrote: On Jun 5, 9:19 am, [EMAIL PROTECTED] (Alvaro Herrera) wrote: Zdenek Kotala wrote: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Is this a TODO? I don't think so; there is no demand from anybody but Zdenek to

[HACKERS] Issues with factorial operator

2007-06-08 Thread Jim C. Nasby
I'm working with a customer that recently discovered that some code had generated the following nice query... SELECT ... WHERE table_id = 92838278! AND ... So their production server now has several processes that are trying to compute some absurdly large factorial. There's two issues here: 1)

Re: [HACKERS] Command tags in create/drop scripts

2007-06-08 Thread Joshua D. Drake
Jim C. Nasby wrote: On Tue, Jun 05, 2007 at 05:52:39PM -, Andrew Hammond wrote: On Jun 5, 9:19 am, [EMAIL PROTECTED] (Alvaro Herrera) wrote: Zdenek Kotala wrote: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Is this a TODO? I don't think so; there is no demand from anybody

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Joshua D. Drake
Alvaro Herrera wrote: Matthew T. O'Connor escribió: Ok, but I think the question posed is that in say a virtual hosting environment there might be say 1,000 databases in the cluster. That is uhmmm insane... 1000 databases? Joshua D. Drake Am I still going to have to wait a long time for

Re: [HACKERS] Issues with factorial operator

2007-06-08 Thread Dann Corbit
It makes sense with factorial function to do an error check on the domain. Calculate beforehand, and figure out what the largest sensible domain value is. For instance, in Maple, I get this: y:=92838278!; Error, object too large The error message returns instantly. For reasonably large

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-08 Thread Dann Corbit
-Original Message- From: [EMAIL PROTECTED] [mailto:pgsql-hackers- [EMAIL PROTECTED] On Behalf Of Joshua D. Drake Sent: Friday, June 08, 2007 10:49 PM To: Alvaro Herrera Cc: Matthew T. O'Connor; Jim C. Nasby; Michael Paesold; Tom Lane; Andrew Hammond; Peter Eisentraut;