Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Alvaro Herrera [EMAIL PROTECTED] wrote:
No, I meant a while (sleep 1(or 10) and counter longtime) check for
exit instead of sleep longtime.
Ah; yes, what I was proposing (or thought about proposing, not sure if I
posted it or not) was putting a upper limit of 10 seconds in the sleep
(bgwriter sleeps 10 seconds if configured to not do anything). Though
10 seconds may seem like an eternity for systems like the ones Peter was
talking about, where there is a script trying to restart the server as
soon as the postmaster dies.
Here is a patch for split-sleep of autovacuum_naptime.
There are some other issues in CVS HEAD; We use the calculation
{autovacuum_naptime * 100} in launcher_determine_sleep().
The result will be corrupted if we set autovacuum_naptime to 2147.
In another place, we use {autovacuum_naptime * 1000}, so we should
set the upper bound to INT_MAX/1000 instead of INT_MAX.
Incidentally, we've already had the same protections for
log_min_duration_statement and log_autovacuum.
I hope this patch could fix those large-autovacuum_naptime problems.
Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center
autovacuum_naptime_overflow.patch
Description: Binary data
---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at
http://www.postgresql.org/about/donate
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Alvaro Herrera wrote: Zeugswetter Andreas ADI SD escribió: The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. This is what we have. The sleep time depends on the schedule of next vacuum for the closest database in time. If naptime is high, the sleep time will be high (depending on number of databases needing attention). No, I meant a while (sleep 1(or 10) and counter longtime) check for exit instead of sleep longtime. Ah; yes, what I was proposing (or thought about proposing, not sure if I posted it or not) was putting a upper limit of 10 seconds in the sleep (bgwriter sleeps 10 seconds if configured to not do anything). Though 10 seconds may seem like an eternity for systems like the ones Peter was talking about, where there is a script trying to restart the server as soon as the postmaster dies. There is also one wild solution. Postmaster and bgwriter will connect with socket/pipe and select command will be used instead sleep. If connection unexpectedly fails, select finish immediately and we are able to handle this issue asap. This socket should be used also in some special case when we need wake up it faster. Zdenek ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
On Tue, Jun 12, 2007 at 12:23:50PM +0200, Zdenek Kotala wrote: Alvaro Herrera wrote: Zeugswetter Andreas ADI SD escribió: The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. This is what we have. The sleep time depends on the schedule of next vacuum for the closest database in time. If naptime is high, the sleep time will be high (depending on number of databases needing attention). No, I meant a while (sleep 1(or 10) and counter longtime) check for exit instead of sleep longtime. Ah; yes, what I was proposing (or thought about proposing, not sure if I posted it or not) was putting a upper limit of 10 seconds in the sleep (bgwriter sleeps 10 seconds if configured to not do anything). Though 10 seconds may seem like an eternity for systems like the ones Peter was talking about, where there is a script trying to restart the server as soon as the postmaster dies. There is also one wild solution. Postmaster and bgwriter will connect with socket/pipe and select command will be used instead sleep. If connection unexpectedly fails, select finish immediately and we are able to handle this issue asap. This socket should be used also in some special case when we need wake up it faster. Given the amount of problems we've had with pipes on win32, let's try to avoid adding extra ones unless they're really necessary. If split-sleep works, that seems a safer bet. //Magnus ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Magnus Hagander wrote: On Tue, Jun 12, 2007 at 12:23:50PM +0200, Zdenek Kotala wrote: Alvaro Herrera wrote: Zeugswetter Andreas ADI SD escribió: The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. This is what we have. The sleep time depends on the schedule of next vacuum for the closest database in time. If naptime is high, the sleep time will be high (depending on number of databases needing attention). No, I meant a while (sleep 1(or 10) and counter longtime) check for exit instead of sleep longtime. Ah; yes, what I was proposing (or thought about proposing, not sure if I posted it or not) was putting a upper limit of 10 seconds in the sleep (bgwriter sleeps 10 seconds if configured to not do anything). Though 10 seconds may seem like an eternity for systems like the ones Peter was talking about, where there is a script trying to restart the server as soon as the postmaster dies. There is also one wild solution. Postmaster and bgwriter will connect with socket/pipe and select command will be used instead sleep. If connection unexpectedly fails, select finish immediately and we are able to handle this issue asap. This socket should be used also in some special case when we need wake up it faster. Given the amount of problems we've had with pipes on win32, let's try to avoid adding extra ones unless they're really necessary. If split-sleep works, that seems a safer bet. Ok It should be problem. But I'm afraid split-sleep is not good solution as well. It should generate a lot of race condition in start/stop scripts and monitoring tools. Much better should be improve pg_ctl to perform clean up (pg_ctl cleanup) when postmaster fails. I think we must offer deterministic way to packagers integrator how to handle this issue. Zdenek ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Matthew T. O'Connor schrieb: Tom Lane wrote: Andrew Hammond [EMAIL PROTECTED] writes: Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be better to say that setting the naptime really high is a Bad Idea. It seems like we should have an upper limit on the GUC variable that's less than INT_MAX ;-). Would an hour be sane? 10 minutes? This is independent of the problem at hand, though, which is that we probably want the launcher to notice postmaster death in less time than autovacuum_naptime, for reasonable values of same. Do we need a configurable autovacuum naptime at all? I know I put it in the original contrib autovacuum because I had no idea what knobs might be needed. I can't see a good reason to ever have a naptime longer than the default 60 seconds, but I suppose one might want a smaller naptime for a very active system? A PostgreSQL database on my laptop for testing. It should use as little resources as possible while being idle. That would be a scenario for naptime greater than 60 seconds, wouldn't it? Best Regards Michael Paesold ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. Andreas ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Zeugswetter Andreas ADI SD escribió: The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. This is what we have. The sleep time depends on the schedule of next vacuum for the closest database in time. If naptime is high, the sleep time will be high (depending on number of databases needing attention). -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Michael Paesold wrote: Matthew T. O'Connor schrieb: Do we need a configurable autovacuum naptime at all? I know I put it in the original contrib autovacuum because I had no idea what knobs might be needed. I can't see a good reason to ever have a naptime longer than the default 60 seconds, but I suppose one might want a smaller naptime for a very active system? A PostgreSQL database on my laptop for testing. It should use as little resources as possible while being idle. That would be a scenario for naptime greater than 60 seconds, wouldn't it? Perhaps, but that isn't the use case PostgresSQL is being designed for. If that is what you really need, then you should probably disable autovacuum. Also a very long naptime means that autovacuum will still wake up at random times and to do the work. At least with short naptime, it will do the work shortly after you updated your tables. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Zeugswetter Andreas ADI SD escribió: The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. This is what we have. The sleep time depends on the schedule of next vacuum for the closest database in time. If naptime is high, the sleep time will be high (depending on number of databases needing attention). No, I meant a while (sleep 1(or 10) and counter longtime) check for exit instead of sleep longtime. Ah; yes, what I was proposing (or thought about proposing, not sure if I posted it or not) was putting a upper limit of 10 seconds in the sleep (bgwriter sleeps 10 seconds if configured to not do anything). Though 10 seconds may seem like an eternity for systems like the ones Peter was talking about, where there is a script trying to restart the server as soon as the postmaster dies. -- Alvaro Herrera Developer, http://www.PostgreSQL.org/ Limítate a mirar... y algun día veras ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
The launcher is set up to wake up in autovacuum_naptime seconds at most. Imho the fix is usually to have a sleep loop. This is what we have. The sleep time depends on the schedule of next vacuum for the closest database in time. If naptime is high, the sleep time will be high (depending on number of databases needing attention). No, I meant a while (sleep 1(or 10) and counter longtime) check for exit instead of sleep longtime. Andreas ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
On Fri, Jun 08, 2007 at 09:49:56AM -0400, Matthew O'Connor wrote: Michael Paesold wrote: Matthew T. O'Connor schrieb: Do we need a configurable autovacuum naptime at all? I know I put it in the original contrib autovacuum because I had no idea what knobs might be needed. I can't see a good reason to ever have a naptime longer than the default 60 seconds, but I suppose one might want a smaller naptime for a very active system? A PostgreSQL database on my laptop for testing. It should use as little resources as possible while being idle. That would be a scenario for naptime greater than 60 seconds, wouldn't it? Perhaps, but that isn't the use case PostgresSQL is being designed for. If that is what you really need, then you should probably disable autovacuum. Also a very long naptime means that autovacuum will still wake up at random times and to do the work. At least with short naptime, it will do the work shortly after you updated your tables. Agreed. Maybe 10 minutes might make sense, but the overhead of checking to see if anything needs vacuuming is pretty tiny. There *is* reason to allow setting the naptime smaller, though (or at least there was; perhaps Alvero's recent changes negate this need): clusters that have a large number of databases. I've worked with folks who are in a hosted environment and give each customer their own database; it's not hard to get a couple hundred databases that way. Setting the naptime higher than a second in such an environment would mean it could be hours before a database is checked for vacuuming. -- Jim Nasby [EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) pgprhXI13515U.pgp Description: PGP signature
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
On Thu, Jun 07, 2007 at 12:13:09PM -0700, Andrew Hammond wrote: On 6/7/07, Jim C. Nasby [EMAIL PROTECTED] wrote: On Mon, Jun 04, 2007 at 11:04:26AM -0400, Alvaro Herrera wrote: The launcher is set up to wake up in autovacuum_naptime seconds at most. So if the user configures a ridiculuos time (for example 86400 seconds, which I've seen) then the launcher would not detect the postmaster death Is there some threshold after which we should have PostgreSQL emit a warning to the effect of autovacuum_naptime is very large. Are you sure you know what you're doing? Yeah, I've seen people set that up with the intention of now autovacuum will only run during our slow time!. I'm thinking it'd be worth mentioning in the docs that this won't work, and instead suggesting that they run vacuumdb -a or equivalent at that time instead. Thoughts? Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be I think we could easily word it so that it's clear that just letting autovacuum do it's thing is preferred. better to say that setting the naptime really high is a Bad Idea. Instead, if they want to shift maintenances to off hours they should consider using a cron job that bonks around the pg_autovacuum.vac_base_thresh or vac_scale_factor values for tables they don't want vacuumed during operational hours (set them really high at the start of operational hours, then to normal during off hours). Tweaking the enable column would work too, but they presumably don't want to disable ANALYZE, although it's entirely likely that new users don't know what ANALYZE does, in which case they _really_ don't want to disable it. That sounds like a rather ugly solution, and one that would be hard to implement; not something to be putting in the docs. -- Jim Nasby [EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) pgpcTAxuATxrP.pgp Description: PGP signature
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Jim C. Nasby escribió: There *is* reason to allow setting the naptime smaller, though (or at least there was; perhaps Alvero's recent changes negate this need): clusters that have a large number of databases. I've worked with folks who are in a hosted environment and give each customer their own database; it's not hard to get a couple hundred databases that way. Setting the naptime higher than a second in such an environment would mean it could be hours before a database is checked for vacuuming. Yes, the code in HEAD is different -- each database will be considered separately. So the huge database taking all day to vacuum will not stop the tiny databases from being vacuumed in a timely manner. And the very huge table in that database will not stop the other tables in the database from being vacuumed either. There can be more than one worker in a single database. The limit is autovacuum_max_workers. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Alvaro Herrera wrote: Jim C. Nasby escribió: There *is* reason to allow setting the naptime smaller, though (or at least there was; perhaps Alvero's recent changes negate this need): clusters that have a large number of databases. I've worked with folks who are in a hosted environment and give each customer their own database; it's not hard to get a couple hundred databases that way. Setting the naptime higher than a second in such an environment would mean it could be hours before a database is checked for vacuuming. Yes, the code in HEAD is different -- each database will be considered separately. So the huge database taking all day to vacuum will not stop the tiny databases from being vacuumed in a timely manner. And the very huge table in that database will not stop the other tables in the database from being vacuumed either. There can be more than one worker in a single database. Ok, but I think the question posed is that in say a virtual hosting environment there might be say 1,000 databases in the cluster. Am I still going to have to wait a long time for my database to get vacuumed? I don't think this has changed much no? (If default naptime is 1 minute, then autovacuum won't even look at a given database but once every 1,000 minutes (16.67 hours) assuming that there isn't enough work to keep all the workers busy.) ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Matthew T. O'Connor escribió: Ok, but I think the question posed is that in say a virtual hosting environment there might be say 1,000 databases in the cluster. Am I still going to have to wait a long time for my database to get vacuumed? I don't think this has changed much no? Depends on how much time it takes to vacuum the other 999 databases. The default max workers is 3. (If default naptime is 1 minute, then autovacuum won't even look at a given database but once every 1,000 minutes (16.67 hours) assuming that there isn't enough work to keep all the workers busy.) The naptime is per database. Which means if you have 1000 databases and a naptime of 60 seconds, the launcher is going to wake up every 100 milliseconds to check things up. (This results from 6 / 1000 = 60 ms, but there is a minimum of 100 ms just to keep things sane). If there are 3 workers and each of the 1000 databases in average takes 10 seconds to vacuum, there will be around 3000 seconds between autovac runs of your database assuming my math is right. I hope those 1000 databases you put in your shared hosting are not very big. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Alvaro Herrera wrote: Matthew T. O'Connor escribió: Ok, but I think the question posed is that in say a virtual hosting environment there might be say 1,000 databases in the cluster. That is uhmmm insane... 1000 databases? Joshua D. Drake Am I still going to have to wait a long time for my database to get vacuumed? I don't think this has changed much no? Depends on how much time it takes to vacuum the other 999 databases. The default max workers is 3. (If default naptime is 1 minute, then autovacuum won't even look at a given database but once every 1,000 minutes (16.67 hours) assuming that there isn't enough work to keep all the workers busy.) The naptime is per database. Which means if you have 1000 databases and a naptime of 60 seconds, the launcher is going to wake up every 100 milliseconds to check things up. (This results from 6 / 1000 = 60 ms, but there is a minimum of 100 ms just to keep things sane). If there are 3 workers and each of the 1000 databases in average takes 10 seconds to vacuum, there will be around 3000 seconds between autovac runs of your database assuming my math is right. I hope those 1000 databases you put in your shared hosting are not very big. -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
-Original Message- From: [EMAIL PROTECTED] [mailto:pgsql-hackers- [EMAIL PROTECTED] On Behalf Of Joshua D. Drake Sent: Friday, June 08, 2007 10:49 PM To: Alvaro Herrera Cc: Matthew T. O'Connor; Jim C. Nasby; Michael Paesold; Tom Lane; Andrew Hammond; Peter Eisentraut; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately Alvaro Herrera wrote: Matthew T. O'Connor escribió: Ok, but I think the question posed is that in say a virtual hosting environment there might be say 1,000 databases in the cluster. That is uhmmm insane... 1000 databases? Not in a test environment. We have several hundred databases here. Of course, only a few dozen (or at most ~100) are of any one type, but I can imagine that under certain circumstances 1000 databases would not be unreasonable. [snip] ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
On Mon, Jun 04, 2007 at 11:04:26AM -0400, Alvaro Herrera wrote: The launcher is set up to wake up in autovacuum_naptime seconds at most. So if the user configures a ridiculuos time (for example 86400 seconds, which I've seen) then the launcher would not detect the postmaster death Yeah, I've seen people set that up with the intention of now autovacuum will only run during our slow time!. I'm thinking it'd be worth mentioning in the docs that this won't work, and instead suggesting that they run vacuumdb -a or equivalent at that time instead. Thoughts? -- Jim Nasby [EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) pgpwir45JckSd.pgp Description: PGP signature
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
On 6/7/07, Jim C. Nasby [EMAIL PROTECTED] wrote: On Mon, Jun 04, 2007 at 11:04:26AM -0400, Alvaro Herrera wrote: The launcher is set up to wake up in autovacuum_naptime seconds at most. So if the user configures a ridiculuos time (for example 86400 seconds, which I've seen) then the launcher would not detect the postmaster death Is there some threshold after which we should have PostgreSQL emit a warning to the effect of autovacuum_naptime is very large. Are you sure you know what you're doing? Yeah, I've seen people set that up with the intention of now autovacuum will only run during our slow time!. I'm thinking it'd be worth mentioning in the docs that this won't work, and instead suggesting that they run vacuumdb -a or equivalent at that time instead. Thoughts? Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be better to say that setting the naptime really high is a Bad Idea. Instead, if they want to shift maintenances to off hours they should consider using a cron job that bonks around the pg_autovacuum.vac_base_thresh or vac_scale_factor values for tables they don't want vacuumed during operational hours (set them really high at the start of operational hours, then to normal during off hours). Tweaking the enable column would work too, but they presumably don't want to disable ANALYZE, although it's entirely likely that new users don't know what ANALYZE does, in which case they _really_ don't want to disable it. This should probably be very close to a section that says something about how insufficient maintenance can be expected to lead to greater performance issues than using autovacuum with default settings. Assuming we believe that to be the case, which I think is reasonable given that we are now defaulting to having autovacuum enabled. Andrew ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Andrew Hammond [EMAIL PROTECTED] writes: Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be better to say that setting the naptime really high is a Bad Idea. It seems like we should have an upper limit on the GUC variable that's less than INT_MAX ;-). Would an hour be sane? 10 minutes? This is independent of the problem at hand, though, which is that we probably want the launcher to notice postmaster death in less time than autovacuum_naptime, for reasonable values of same. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Tom Lane wrote: Andrew Hammond [EMAIL PROTECTED] writes: Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be better to say that setting the naptime really high is a Bad Idea. It seems like we should have an upper limit on the GUC variable that's less than INT_MAX ;-). Would an hour be sane? 10 minutes? This is independent of the problem at hand, though, which is that we probably want the launcher to notice postmaster death in less time than autovacuum_naptime, for reasonable values of same. Do we need a configurable autovacuum naptime at all? I know I put it in the original contrib autovacuum because I had no idea what knobs might be needed. I can't see a good reason to ever have a naptime longer than the default 60 seconds, but I suppose one might want a smaller naptime for a very active system? ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Best Practice for running vacuums during off hours WAS Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/7/07, Matthew T. O'Connor wrote: Tom Lane wrote: Andrew Hammond writes: Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be better to say that setting the naptime really high is a Bad Idea. It seems like we should have an upper limit on the GUC variable that's less than INT_MAX ;-). Would an hour be sane? 10 minutes? This is independent of the problem at hand, though, which is that we probably want the launcher to notice postmaster death in less time than autovacuum_naptime, for reasonable values of same. Do we need a configurable autovacuum naptime at all? I know I put it in the original contrib autovacuum because I had no idea what knobs might be needed. I can't see a good reason to ever have a naptime longer than the default 60 seconds, but I suppose one might want a smaller naptime for a very active system? That's a good question. I can't see any reason for a naptime longer than 60 seconds either. I think very large naptime settings are a symptom of another issue: what's the Right Way to defer vacuums until off hours? Is that even a desirable thing anymore? I don't think it is in the majority of cases. I originally thought that this was more of a Best Practices issue (ie, fix in the docs, not the code), but now I'm wondering if there's much call for supporting the idea of being more aggressive with vacuums at different times of the day / week / month. Anyone? -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (Darwin) iD8DBQFGaIdN+zlEYLc6JJgRAiNFAJ49CQwiTVxWhXNeOzIBABLN5LZY3wCfUj/W ZLakjPyRVwOijaB6keS3ld8= =Hg/X -END PGP SIGNATURE- ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: Best Practice for running vacuums during off hours WAS Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Andrew Hammond escribió: That's a good question. I can't see any reason for a naptime longer than 60 seconds either. I think very large naptime settings are a symptom of another issue: what's the Right Way to defer vacuums until off hours? Is that even a desirable thing anymore? I don't think it is in the majority of cases. I originally thought that this was more of a Best Practices issue (ie, fix in the docs, not the code), but now I'm wondering if there's much call for supporting the idea of being more aggressive with vacuums at different times of the day / week / month. Anyone? That's why the intention is to have the autovacuum scheduling feature be a mechanism for changing the autovac parameters according to date and time. We even have a Google Summer of Code project about that. -- Alvaro Herrera http://www.amazon.com/gp/registry/DXLWNGRJD34J La virtud es el justo medio entre dos defectos (Aristóteles) ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately
Peter Eisentraut wrote: I notice that in 8.3, when I kill the postmaster process with SIGKILL or SIGSEGV, the child processes writer and stats collector go away immediately, but the autovacuum launcher hangs around for up to a minute. (I suppose this has to do with the periodic wakeups?). When you try to restart the postmaster before that it fails with a complaint that someone is still attached to the shared memory segment. These are obviously not normal modes of operation, but I fear that this could cause some problems with people's control scripts of the sort, it crashed, let's try to restart it. The launcher is set up to wake up in autovacuum_naptime seconds at most. So if the user configures a ridiculuos time (for example 86400 seconds, which I've seen) then the launcher would not detect the postmaster death for a very long time, which is probably bad. (You measured a one minute delay because that's the default naptime). Maybe this is not such a hot idea, and we should wake the launcher up every 10 seconds (or less?). I picked 10 seconds because that's the time the bgwriter sleeps if there is no activity configured. Does this sound acceptable? The only problem with waking it up too frequently is that it would be waking the system up (for gettimeofday()) even if nothing is happening. I also just noticed that the launcher will check if postmaster is alive, then sleep, and then possibly do some work. So if the postmaster died in the sleep period, the launcher might try to do some work. Should we add a check for postmaster liveliness after the sleep? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
