Re: [PATCHES] Fix for erroneous warning on Shutdown
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> ... The question stands though: why isn't it > >> appropriate to warn of overly-frequently-issued manual checkpoints? > > > ... the warning is for cases when you are filling up the WAL logs too > > quickly and checkpoints are happening too frequently. If a user is > > doing checkpoints, it isn't anything increasing the checkpoint segments > > is going to help. > > No, I think the warning is for when checkpoints are happening too > frequently, period. An overly small checkpoint_segments setting > is one possible cause of that, but the performance penalty from > too many checkpoints is just as bad no matter what's causing it. > (Remember that a checkpoint not only forces I/O in itself, but > significantly increases subsequent WAL traffic because of needing > to dump whole page images into WAL.) > > How do you feel about improving the signaling mechanism but using > it just to vary the HINT? > > LOG: checkpoints are occurring too frequently (nn seconds apart) > HINT: Consider increasing the configuration parameter "checkpoint_segments". > > LOG: checkpoints are occurring too frequently (nn seconds apart) > HINT: Issuing explicit CHECKPOINTs so often is really expensive. Sure, fine by me. My only point is that we need something to tell people they need to increase their checkpoint_segments. If we add other warnings, that is fine too. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] Fix for erroneous warning on Shutdown
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> ... The question stands though: why isn't it >> appropriate to warn of overly-frequently-issued manual checkpoints? > ... the warning is for cases when you are filling up the WAL logs too > quickly and checkpoints are happening too frequently. If a user is > doing checkpoints, it isn't anything increasing the checkpoint segments > is going to help. No, I think the warning is for when checkpoints are happening too frequently, period. An overly small checkpoint_segments setting is one possible cause of that, but the performance penalty from too many checkpoints is just as bad no matter what's causing it. (Remember that a checkpoint not only forces I/O in itself, but significantly increases subsequent WAL traffic because of needing to dump whole page images into WAL.) How do you feel about improving the signaling mechanism but using it just to vary the HINT? LOG: checkpoints are occurring too frequently (nn seconds apart) HINT: Consider increasing the configuration parameter "checkpoint_segments". LOG: checkpoints are occurring too frequently (nn seconds apart) HINT: Issuing explicit CHECKPOINTs so often is really expensive. and so on. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Fix for erroneous warning on Shutdown
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> I could argue that a client-driven process that issues CHECKPOINT every > >> few seconds is equally deserving of a warning. The only thing wrong is > >> that the HINT is inapplicable ... but that's why it's a HINT and not > >> part of the main message. > > > Also consider they could have issued a checkpoint right after the system > > did one. Yuck. > > > When I added the warning I hoped to only have it happen for full logs > > and not CHECKPOINT, but I guess I couldn't and someone else realized > > that and added that clearer comment, or originally I could do that, but > > since it has been moved into the bgwriter, it can't anymore. > > I believe the original implementation in the postmaster had a somewhat > different set of bugs ;-). IIRC it did not react to manual checkpoints > but it did confuse WAL checkpoints with timeout-driven checkpoints. > The present bgwriter can distinguish the third but not the first two. > > If we were willing to take the time to generalize the > backend-to-bgwriter signaling mechanism then we could distinguish > WAL-driven checkpoints from manually issued checkpoints. I'm sort of > intending to do that anyway. The question stands though: why isn't it > appropriate to warn of overly-frequently-issued manual checkpoints? Imagine pgbench issuing a checkpoint, which it does. That could trigger the warning when the parameter really shouldn't be increased. To me, the warning is for cases when you are filling up the WAL logs too quickly and checkpoints are happening too frequently. If a user is doing checkpoints, it isn't anything increasing the checkpoint segments is going to help. If you can't fix the code to distinguish between manual and wal-full checkpoints, we can adjust the warning logic to warn when there are 3 checkpoints in a short period, rather than just two. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] Fix for erroneous warning on Shutdown
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I could argue that a client-driven process that issues CHECKPOINT every >> few seconds is equally deserving of a warning. The only thing wrong is >> that the HINT is inapplicable ... but that's why it's a HINT and not >> part of the main message. > Also consider they could have issued a checkpoint right after the system > did one. Yuck. > When I added the warning I hoped to only have it happen for full logs > and not CHECKPOINT, but I guess I couldn't and someone else realized > that and added that clearer comment, or originally I could do that, but > since it has been moved into the bgwriter, it can't anymore. I believe the original implementation in the postmaster had a somewhat different set of bugs ;-). IIRC it did not react to manual checkpoints but it did confuse WAL checkpoints with timeout-driven checkpoints. The present bgwriter can distinguish the third but not the first two. If we were willing to take the time to generalize the backend-to-bgwriter signaling mechanism then we could distinguish WAL-driven checkpoints from manually issued checkpoints. I'm sort of intending to do that anyway. The question stands though: why isn't it appropriate to warn of overly-frequently-issued manual checkpoints? regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] Fix for erroneous warning on Shutdown
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > I just did CHECKPOINT;CHECKPOINT and got the warning in the logs. This > > needs to be fixed. > > See code: > > /* > * Ideally we should only warn if this checkpoint was > * requested due to running out of segment files, and not > * if it was manually requested. However we can't tell the > * difference with the current signalling mechanism. > */ > > I could argue that a client-driven process that issues CHECKPOINT every > few seconds is equally deserving of a warning. The only thing wrong is > that the HINT is inapplicable ... but that's why it's a HINT and not > part of the main message. Also consider they could have issued a checkpoint right after the system did one. Yuck. When I added the warning I hoped to only have it happen for full logs and not CHECKPOINT, but I guess I couldn't and someone else realized that and added that clearer comment, or originally I could do that, but since it has been moved into the bgwriter, it can't anymore. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PATCHES] Fix for erroneous warning on Shutdown
Bruce Momjian <[EMAIL PROTECTED]> writes: > I just did CHECKPOINT;CHECKPOINT and got the warning in the logs. This > needs to be fixed. See code: /* * Ideally we should only warn if this checkpoint was * requested due to running out of segment files, and not * if it was manually requested. However we can't tell the * difference with the current signalling mechanism. */ I could argue that a client-driven process that issues CHECKPOINT every few seconds is equally deserving of a warning. The only thing wrong is that the HINT is inapplicable ... but that's why it's a HINT and not part of the main message. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] Fix for erroneous warning on Shutdown
Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Shutdown was via CTRL-Cmake a difference? > > Wouldn't think so. > > I can force the message to appear if I do a *manual* CHECKPOINT command > within thirty seconds of startup. I'm not sure if that should be > considered wrong or not, but in any case it doesn't seem to be what > you're describing. It is wrong. The message should not appear if you issue CHECKPOINT. I just did CHECKPOINT;CHECKPOINT and got the warning in the logs. This needs to be fixed. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] PITR Archival
Simon Riggs wrote: > Also note: archive_mode is not designed to be turned on/off frequently. > It is possible to confuse it if you turn it on, then restart with it > off, then turn it on again. That is likely to create a "hole" in the > archive history of xlogs and you will not be able to recover correctly. I assume full xlog files will still be transfered, but that you will be missing some files while the archiver was turned off. That issue will be part of the restore procedure, I assume. I think we need to specify people should use an empty archive directory every time they turn on archiving. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] PITR Archival
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches I will try to apply it once it is reviewed. --- Simon Riggs wrote: > I enclose a working set of context diff patches and new files to make > PITR archiving work, as of cvstip (NOW). > > You'll see the new options in the postgresql.conf...though you may wish > to use archive_debug = true as well, when testing. > > There is one bug: shutdown doesn't work quite right. I haven't fixed > this because I've spent too long trying to decipher how pgstat did a > clean shutdown, discovering now that it didn't and that has now been > patched...something similar is required for pgarch, but I'm out of time > now...leaving time for discussion of this lot... > > I'm looking to have this lot committed asap, cos my fingers are starting > to catch the bitrot now. :) > > I have a considerable amount still to learn about CVS, diff and patch, > so anybody wanting to spend 10-15 mins on the phone with me would > greatly enhance my chances of helping patch my patch, when the bugs roll > in. > > If we get this done smoothly, I reckon I can have some PITR recovery > control done by beta freeze. > > Best regards, Simon Riggs [ Attachment, skipping... ] [ Attachment, skipping... ] [ Attachment, skipping... ] > > ---(end of broadcast)--- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to [EMAIL PROTECTED] so that your > message can get through to the mailing list cleanly -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] stderr & win32 admin check
> At least I don't htink it's in what Claudio has so far - Claudio? Lots > of work to get into your framework? The original patch I submitted actually *required* a service name, allowing any number of postgres installations. I have no intention of removing that. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see http://www.memetrics.com/emailpolicy.html";>http://www.memetrics.com/em ailpolicy.html ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PATCHES] stderr & win32 admin check
Tom Lane wrote: > Andrew Dunstan <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> So? I don't follow why "run it as a service" isn't a sufficient answer, > >> and indeed the preferred way to do it. > > > We don't know what the usage pattern is going to be on Windows - I think > > we need to keep it as flexible as possible consistent with good > > security. > > Sure, but I draw the line at running Postgres with admin privileges. > "Flexibility is more important than security" is exactly the mindset > that has gotten Microsoft into their current bed of nails. Easy of use is also one of the reaons MS is so popular and on so many computers. I am not advocating we allow unsecure usage, but I am pointing out that security and easy of use are important. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] stderr & win32 admin check
Dave Page wrote: > > -Original Message- > From: Matthew T. O'Connor [mailto:[EMAIL PROTECTED] > Sent: Tue 6/15/2004 4:06 PM > To: Dave Page > Cc: Tom Lane; Magnus Hagander; [EMAIL PROTECTED] > Subject: Re: [PATCHES] stderr & win32 admin check > > > I have been working on integrating pg_autovacuum into the backend, and I > > have it working, I'm just trying to clean up some lose ends before I > > submit another patch. I think backend integration will eliminate the > > need for your autovacuum service patch no? > > Oh, of course. I wrote it 'just in case', and for a bit of experience before looking > at the main server. Which Claudio then did anyway! OK, I have removed your patches on this from my mailbox. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] stderr & win32 admin check
>>>It hasn't been discussed, but it would be fairly trivial to >>>add this to the service installer. (A bit more work on the MSI >>>installer, but we could do with that one just installing the >>>default instance at least for starters). >> >> Correcting myself on this one - the MSI installer already supports >> instaling under any name you want. Only one, but you can >pick the name >> (default being "postgresql "). >> Since the service code is not in th ebackend yet, you can't actually >> install this, but the interface is there and ready. >> > >I have often seen multiple instances of a service under different names >(think of srvany from the Resources Toolkit). > >I agree that the serice name should be configurable. > >Will the installer have a set of options? e.g. >. install binaries >. initdb >. install/configure service? Yes. >If so, maybe you could just run it again, missing out the >install binaries >step, to create another service instance. Well, ATM it will detect that it's already installed, and offer to remove the program. But that can be changed to allow just this, yes. What it will not allow (per MSI specs AFAIK) is to install the same version in different places. But it can still install multiple services calling the same postmaster.exe with different arguments. I think :-) //Magnus ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] stderr & win32 admin check
Tom Lane said: > Andrew Dunstan <[EMAIL PROTECTED]> writes: >> Tom Lane wrote: >>> So? I don't follow why "run it as a service" isn't a sufficient >>> answer, and indeed the preferred way to do it. > >> We don't know what the usage pattern is going to be on Windows - I >> think we need to keep it as flexible as possible consistent with good >> security. > > Sure, but I draw the line at running Postgres with admin privileges. > "Flexibility is more important than security" is exactly the mindset > that has gotten Microsoft into their current bed of nails. > That would be the reason for my qualifying phrase "consistent with good security". :-) cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] stderr & win32 admin check
Magnus Hagander said: >>It hasn't been discussed, but it would be fairly trivial to >>add this to the service installer. (A bit more work on the MSI >>installer, but we could do with that one just installing the >>default instance at least for starters). > > Correcting myself on this one - the MSI installer already supports > instaling under any name you want. Only one, but you can pick the name > (default being "postgresql "). > Since the service code is not in th ebackend yet, you can't actually > install this, but the interface is there and ready. > I have often seen multiple instances of a service under different names (think of srvany from the Resources Toolkit). I agree that the serice name should be configurable. Will the installer have a set of options? e.g. . install binaries . initdb . install/configure service? If so, maybe you could just run it again, missing out the install binaries step, to create another service instance. cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] stderr & win32 admin check
>It hasn't been discussed, but it would be fairly trivial to >add this to the service installer. (A bit more work on the MSI >installer, but we could do with that one just installing the >default instance at least for starters). Correcting myself on this one - the MSI installer already supports instaling under any name you want. Only one, but you can pick the name (default being "postgresql "). Since the service code is not in th ebackend yet, you can't actually install this, but the interface is there and ready. //Magnus ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] stderr & win32 admin check
>> If you mean only run one instance of postmaster as service, >> that's not true. >> If you like two pgsql servers (i.e. db clusters), you can >> install two services, both using the same binary with >> different cmd line arguments. > >In which case, what would 'net stop postgresql' do? What you suggest is >correct if using something like the Cygwin service installer >because you >can specify unique service names etc. AFAIK there are no plans to allow >such configurability in PostgreSQL - nor is that something you see in >most other services. 'net stop postgresql' would stop the instance named 'postgresql'. 'net stop someotherpostgresql' would stop the instance named 'someotherpostgresql'. It hasn't been discussed, but it would be fairly trivial to add this to the service installer. (A bit more work on the MSI installer, but we could do with that one just installing the default instance at least for starters). At least I don't htink it's in what Claudio has so far - Claudio? Lots of work to get into your framework? And FWIW, that's exactly what Microsoft SQL Server does when you install multiple instances on the same machine. //Magnus ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PATCHES] stderr & win32 admin check
> -Original Message- > From: Andreas Pflug [mailto:[EMAIL PROTECTED] > Sent: 15 June 2004 22:28 > To: Dave Page > Cc: Tom Lane; [EMAIL PROTECTED] > Subject: Re: [PATCHES] stderr & win32 admin check > > Dave Page wrote: > > > > > > > > > you can only run one > >instance as a service on a single machine. > > > > If you mean only run one instance of postmaster as service, > that's not true. > If you like two pgsql servers (i.e. db clusters), you can > install two services, both using the same binary with > different cmd line arguments. In which case, what would 'net stop postgresql' do? What you suggest is correct if using something like the Cygwin service installer because you can specify unique service names etc. AFAIK there are no plans to allow such configurability in PostgreSQL - nor is that something you see in most other services. Regards, Dave. ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] stderr & win32 admin check
Dave Page wrote: you can only run one instance as a service on a single machine. If you mean only run one instance of postmaster as service, that's not true. If you like two pgsql servers (i.e. db clusters), you can install two services, both using the same binary with different cmd line arguments. Regards, Andreas ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] stderr & win32 admin check
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> So? I don't follow why "run it as a service" isn't a sufficient answer, >> and indeed the preferred way to do it. > We don't know what the usage pattern is going to be on Windows - I think > we need to keep it as flexible as possible consistent with good > security. Sure, but I draw the line at running Postgres with admin privileges. "Flexibility is more important than security" is exactly the mindset that has gotten Microsoft into their current bed of nails. The fact that there is a perfectly usable solution on NT4 (the oldest Windows version we have any intention of supporting) seems enough to me. There are more usable solutions on newer versions. Fine. But nowhere in here do I see a sufficient reason to allow known-insecure operating practices. I might be more willing to listen to other opinions on this if I were rejecting a somewhat smaller volume of Microsoft-security-hole-spawned spam and viruses every day. But in the current environment I don't see how any sane person can argue that allowing insecure operation of a network-exposed service is acceptable behavior. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] stderr & win32 admin check
"Dave Page" <[EMAIL PROTECTED]> writes: >> So? I don't follow why "run it as a service" isn't a >> sufficient answer, and indeed the preferred way to do it. > It is the preferred method, however two reasons not to spring to mind: > first, I bet you and most others on this list might not want to do that > if forced to develop on a Windows box, and second, you can only run one > instance as a service on a single machine. If forced to develop on Windows, I certainly wouldn't hold still for developing on NT4. This argument would have more force if it were referring to a remotely current release, but that one is four Windows revs back for pete's sake. It's not going to have every feature of newer releases. This is one. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] Fix for erroneous warning on Shutdown
On Tue, 2004-06-15 at 21:04, Tom Lane wrote: > Andrew Dunstan <[EMAIL PROTECTED]> writes: > > Just to confirm that Simon is not suffering this uniquely, I saw this > > the other day on Windows, I believe - meant to report it but it got away > > from me. > > Oh, I bet I know what's going on --- are you guys launching the > postmaster in a console window and then typing ^C to shut it down? > Depending on your shell that might result in SIGINT being delivered > not only to the postmaster but to all its children. Backends, if > any, will take that as a query-cancel while the bgwriter will take > it as a checkpoint request. Which could easily result in the > too-frequent-checkpoints message. > > The only real fix I can see for this is to use some other signal than > SIGINT to transmit checkpoint requests to the bgwriter. Maybe we could > piggyback all bgwriter requests onto SIGUSR2, comparable to the way that > signals up to the postmaster are now handled. Not sure how important > it is though. > Sounds like the cause to me...that would explain why it has just started happening... Its minor then... I would only do that for testing... Regards, Simon ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] stderr & win32 admin check
> -Original Message- > From: Tom Lane [mailto:[EMAIL PROTECTED] > Sent: 15 June 2004 19:11 > To: Dave Page > Cc: Magnus Hagander; [EMAIL PROTECTED] > Subject: Re: [PATCHES] stderr & win32 admin check > > So? I don't follow why "run it as a service" isn't a > sufficient answer, and indeed the preferred way to do it. It is the preferred method, however two reasons not to spring to mind: first, I bet you and most others on this list might not want to do that if forced to develop on a Windows box, and second, you can only run one instance as a service on a single machine. Like I said though, I run XP so it's not a problem for me - I'm just sticking up for the NT4 users it was decided we should support. Regards, Dave. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] Fix for erroneous warning on Shutdown
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Just to confirm that Simon is not suffering this uniquely, I saw this > the other day on Windows, I believe - meant to report it but it got away > from me. Oh, I bet I know what's going on --- are you guys launching the postmaster in a console window and then typing ^C to shut it down? Depending on your shell that might result in SIGINT being delivered not only to the postmaster but to all its children. Backends, if any, will take that as a query-cancel while the bgwriter will take it as a checkpoint request. Which could easily result in the too-frequent-checkpoints message. The only real fix I can see for this is to use some other signal than SIGINT to transmit checkpoint requests to the bgwriter. Maybe we could piggyback all bgwriter requests onto SIGUSR2, comparable to the way that signals up to the postmaster are now handled. Not sure how important it is though. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] Fix for erroneous warning on Shutdown
Tom Lane wrote: Simon Riggs <[EMAIL PROTECTED]> writes: Shutdown was via CTRL-Cmake a difference? Wouldn't think so. I can force the message to appear if I do a *manual* CHECKPOINT command within thirty seconds of startup. I'm not sure if that should be considered wrong or not, but in any case it doesn't seem to be what you're describing. Just to confirm that Simon is not suffering this uniquely, I saw this the other day on Windows, I believe - meant to report it but it got away from me. cheers andrew ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Fix for erroneous warning on Shutdown
Simon Riggs <[EMAIL PROTECTED]> writes: > Shutdown was via CTRL-Cmake a difference? Wouldn't think so. I can force the message to appear if I do a *manual* CHECKPOINT command within thirty seconds of startup. I'm not sure if that should be considered wrong or not, but in any case it doesn't seem to be what you're describing. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] Fix for erroneous warning on Shutdown
On Tue, 2004-06-15 at 19:33, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > As of now, (i.e. even including the new bgwriter shutdown) if you: > > 1. start postmaster > > 2. do some work that writes xlog > > 3. shutdown within some few seconds of startup > > you get a WARNING suggesting you increase CHECKPOINT_SEGMENTS, which is > > clearly erroneous since no checkpoint has taken place since startup. > > [ scratches head... ] Not for me. Are you using any nondefault > settings in postgresql.conf? Hmmm...I did a complete re-download of CVS, then a vanilla configure/make/make install with .conf straight from can. I wanted to be absolutely certain I had got it clean. Shutdown was via CTRL-Cmake a difference? Damn I hate the wierd ones...if only because of the 90% chance its something wrong that I've done... I'll do it again more slowly and report back... Best Regards, Simon Riggs ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PATCHES] Fix for erroneous warning on Shutdown
Simon Riggs <[EMAIL PROTECTED]> writes: > As of now, (i.e. even including the new bgwriter shutdown) if you: > 1. start postmaster > 2. do some work that writes xlog > 3. shutdown within some few seconds of startup > you get a WARNING suggesting you increase CHECKPOINT_SEGMENTS, which is > clearly erroneous since no checkpoint has taken place since startup. [ scratches head... ] Not for me. Are you using any nondefault settings in postgresql.conf? regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] stderr & win32 admin check
Tom Lane wrote: "Dave Page" <[EMAIL PROTECTED]> writes: Well, that's kinda the point. If you are a hacker who has local admin privs (not exactly unusual on Windows networks - in some cases Power User group membership is required to run legacy software), you *cannot* run PostgreSQL except as a service, thus potentially making it a show stopper for those users. So? I don't follow why "run it as a service" isn't a sufficient answer, and indeed the preferred way to do it. We don't know what the usage pattern is going to be on Windows - I think we need to keep it as flexible as possible consistent with good security. Bear in mind that Windows is effectively a one-console system - that's one reason why so many users just log on as Administrator - switching to another user is much more of a pain than it is on Unix. cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PATCHES] stderr & win32 admin check
"Dave Page" <[EMAIL PROTECTED]> writes: > Well, that's kinda the point. If you are a hacker who has local admin > privs (not exactly unusual on Windows networks - in some cases Power > User group membership is required to run legacy software), you *cannot* > run PostgreSQL except as a service, thus potentially making it a show > stopper for those users. So? I don't follow why "run it as a service" isn't a sufficient answer, and indeed the preferred way to do it. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] PITR Archival
On Tue, Jun 15, 2004 at 04:34:30PM +0100, Simon Riggs wrote: > I have a considerable amount still to learn about CVS, diff and patch, > so anybody wanting to spend 10-15 mins on the phone with me would > greatly enhance my chances of helping patch my patch, when the bugs roll > in. Can't help you with the phone thingie, but if you want to see what's in a patch and be able to edit it nicely, I suggest you use meld. It's a python dual-pane GTK display, really nice. Too slow for real coding (on my machine that is) but real good for seeing what you changed. Watch out for use of Ctrl, Shift and Alt -- very handy. Can use CVS as well, or you can use two source trees. -- Alvaro Herrera () "There is evil in the world. There are dark, awful things. Occasionally, we get a glimpse of them. But there are dark corners; horrors almost impossible to imagine... even in our worst nightmares." (Van Helsing, Dracula A.D. 1972) ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] stderr & win32 admin check
Dave Page wrote: It could still be run on NT4 under the following conditions: 1) Running as a service 2) Running if the user logged in is not an administrator. Well, isn't "running as a service" sufficient? I thought that was the only interesting case for non-hackers anyway. As long as you get an error message that's reasonably clear about what you can do instead, this hardly seems like a showstopper... Well, that's kinda the point. If you are a hacker who has local admin privs (not exactly unusual on Windows networks - in some cases Power User group membership is required to run legacy software), Not only legacy software... you *cannot* run PostgreSQL except as a service, thus potentially making it a show stopper for those users. Actually I wouldn't expect a server to run as anything else but a service. Running a server from a command line is for debugging purposes only (in the win32 world). Thus I'd consider the non-admin check as acceptable, while quite irritating for most win32 users. Many win32 servers create an own account on installation or suggest to do so (instead of using the "Local System" account), this could make things more convenient for the average win32 user. Regards, Andreas Regards, Andreas ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] PITR Archival
On Tue, 2004-06-15 at 16:34, Simon Riggs wrote: > I enclose a working set of context diff patches and new files to make > PITR archiving work, as of cvstip (NOW). > > You'll see the new options in the postgresql.conf...though you may wish > to use archive_debug = true as well, when testing. > > There is one bug: shutdown doesn't work quite right. I haven't fixed > this because I've spent too long trying to decipher how pgstat did a > clean shutdown, discovering now that it didn't and that has now been > patched...something similar is required for pgarch, but I'm out of time > now...leaving time for discussion of this lot... > The patch creates a further child process of postmaster, the archiver. Archiver code is similar, but not the same, as pgstat.c, and by analogy lives in src/backend/postmaster/pgarch.c with a matching pgarch.h in src/include (just as with pgstat.h) At various points, 5 processes are involved, all of which are identified clearly in the debugging messages... - backend: sends signal to postmaster to say xlog is ready to archive - postmaster: catches and resends to archiver - archiver: then calls system(3) to invoke a user-defined archival task - bgwriter: later cleans up archive_status at checkpoint time Please note: you will need to initdb to make this work Also note: archive_mode is not designed to be turned on/off frequently. It is possible to confuse it if you turn it on, then restart with it off, then turn it on again. That is likely to create a "hole" in the archive history of xlogs and you will not be able to recover correctly. You may also note that the design has changed substantially from many earlier design postings...all of this is based on community input to rationalise behaviour and to streamline code. If you wish to test recovery, you should: - do a full physical backup of DataDir, while postmaster is UP - archive all xlogs when disaster strikes - restore backup of DataDir - restore all archived xlogs to pg_xlog - startup postmaster (and watch...) Further work is still required to make it STOP recovering at a predetermined point in time Best regards, Simon Riggs ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] Fix for erroneous warning on Shutdown
On Fri, 2004-06-11 at 19:25, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Minor patch to correct erroneous warning in cvs tip, believed to be a > > very minor regression. > > This patch is wrong; it effectively disables the warning altogether. > > > When a shutdown was requested within CHECKPOINT_SECONDS of a checkpoint, > > the shutdown code in the bgwriter (which has does checkpointing) would > > issue the erroneous message: > > I don't think so ... the shutdown path doesn't go through this code. > Could you take another look at the cause of what you saw? Hmmm...perhaps I should submit this as a *low priority* bug then, rather than just a patch? As of now, (i.e. even including the new bgwriter shutdown) if you: 1. start postmaster 2. do some work that writes xlog 3. shutdown within some few seconds of startup you get a WARNING suggesting you increase CHECKPOINT_SEGMENTS, which is clearly erroneous since no checkpoint has taken place since startup. Not exactly a common circumstance, I grant you, but it does seem to be a regression nonetheless. My thinking was that the circumstance was not limited to this edge case, but actually any shutdown within 30s of a checkpoint, but the latter is just speculation. Only reason I spotted it is I've been doing a lot of startup/shutdown work recently... Best Regards, Simon Riggs ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] stderr & win32 admin check
-Original Message- From: Matthew T. O'Connor [mailto:[EMAIL PROTECTED] Sent: Tue 6/15/2004 4:06 PM To: Dave Page Cc: Tom Lane; Magnus Hagander; [EMAIL PROTECTED] Subject: Re: [PATCHES] stderr & win32 admin check > I have been working on integrating pg_autovacuum into the backend, and I > have it working, I'm just trying to clean up some lose ends before I > submit another patch. I think backend integration will eliminate the > need for your autovacuum service patch no? Oh, of course. I wrote it 'just in case', and for a bit of experience before looking at the main server. Which Claudio then did anyway! Regards, Dave ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
[PATCHES] PITR Archival
I enclose a working set of context diff patches and new files to make
PITR archiving work, as of cvstip (NOW).
You'll see the new options in the postgresql.conf...though you may wish
to use archive_debug = true as well, when testing.
There is one bug: shutdown doesn't work quite right. I haven't fixed
this because I've spent too long trying to decipher how pgstat did a
clean shutdown, discovering now that it didn't and that has now been
patched...something similar is required for pgarch, but I'm out of time
now...leaving time for discussion of this lot...
I'm looking to have this lot committed asap, cos my fingers are starting
to catch the bitrot now. :)
I have a considerable amount still to learn about CVS, diff and patch,
so anybody wanting to spend 10-15 mins on the phone with me would
greatly enhance my chances of helping patch my patch, when the bugs roll
in.
If we get this done smoothly, I reckon I can have some PITR recovery
control done by beta freeze.
Best regards, Simon Riggs
? Makefile.global
? pitr_arch_v3.01.patch
? pitr_v3.1.patch
? backend/postgres
? backend/catalog/postgres.bki
? backend/catalog/postgres.description
? backend/postmaster/pgarch.c
? backend/postmaster/postmaster.c.new
? backend/utils/mb/conversion_procs/conversion_create.sql
? backend/utils/mb/conversion_procs/ascii_and_mic/libascii_and_mic.so.0.0
? backend/utils/mb/conversion_procs/cyrillic_and_mic/libcyrillic_and_mic.so.0.0
? backend/utils/mb/conversion_procs/euc_cn_and_mic/libeuc_cn_and_mic.so.0.0
? backend/utils/mb/conversion_procs/euc_jp_and_sjis/libeuc_jp_and_sjis.so.0.0
? backend/utils/mb/conversion_procs/euc_kr_and_mic/libeuc_kr_and_mic.so.0.0
? backend/utils/mb/conversion_procs/euc_tw_and_big5/libeuc_tw_and_big5.so.0.0
? backend/utils/mb/conversion_procs/latin2_and_win1250/liblatin2_and_win1250.so.0.0
? backend/utils/mb/conversion_procs/latin_and_mic/liblatin_and_mic.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_ascii/libutf8_and_ascii.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_big5/libutf8_and_big5.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_cyrillic/libutf8_and_cyrillic.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_euc_cn/libutf8_and_euc_cn.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_euc_jp/libutf8_and_euc_jp.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_euc_kr/libutf8_and_euc_kr.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_euc_tw/libutf8_and_euc_tw.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_gb18030/libutf8_and_gb18030.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_gbk/libutf8_and_gbk.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_iso8859/libutf8_and_iso8859.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_iso8859_1/libutf8_and_iso8859_1.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_johab/libutf8_and_johab.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_sjis/libutf8_and_sjis.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_tcvn/libutf8_and_tcvn.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_uhc/libutf8_and_uhc.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_win1250/libutf8_and_win1250.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_win1256/libutf8_and_win1256.so.0.0
? backend/utils/mb/conversion_procs/utf8_and_win874/libutf8_and_win874.so.0.0
? bin/initdb/initdb
? bin/initlocation/initlocation
? bin/ipcclean/ipcclean
? bin/pg_config/pg_config
? bin/pg_controldata/pg_controldata
? bin/pg_ctl/pg_ctl
? bin/pg_dump/pg_dump
? bin/pg_dump/pg_dumpall
? bin/pg_dump/pg_restore
? bin/pg_resetxlog/pg_resetxlog
? bin/psql/psql
? bin/scripts/clusterdb
? bin/scripts/createdb
? bin/scripts/createlang
? bin/scripts/createuser
? bin/scripts/dropdb
? bin/scripts/droplang
? bin/scripts/dropuser
? bin/scripts/vacuumdb
? include/pg_config.h
? include/pgarch.h
? include/stamp-h
? interfaces/ecpg/compatlib/libecpg_compat.so.1.1
? interfaces/ecpg/ecpglib/libecpg.so.4.2
? interfaces/ecpg/pgtypeslib/libpgtypes.so.1.2
? interfaces/ecpg/preproc/ecpg
? interfaces/libpq/libpq.so.3.2
? pl/plpgsql/src/libplpgsql.so.1.0
? port/pg_config_paths.h
? timezone/zic
Index: backend/access/nbtree/nbtsort.c
===
RCS file: /projects/cvsroot/pgsql-server/src/backend/access/nbtree/nbtsort.c,v
retrieving revision 1.82
diff -c -r1.82 nbtsort.c
*** backend/access/nbtree/nbtsort.c 2 Jun 2004 17:28:17 - 1.82
--- backend/access/nbtree/nbtsort.c 15 Jun 2004 15:19:48 -
***
*** 67,72
--- 67,73
#include "miscadmin.h"
#include "storage/smgr.h"
#include "utils/tuplesort.h"
+ #include "access/xlog.h"
/*
***
*** 220,235
wstate.index = btspool->index;
/*
! * We need to log index creation in WAL iff WAL archiving is enabled
* AND it's not a temp index.
- *
- * XXX when WAL archiving is actually supported, this test will likely
- * need to change; and the hardwired extern is cruddy anyway ...
*/
{
! extern char XLOG_archiv
Re: [PATCHES] stderr & win32 admin check
Dave Page wrote: Personally I don't care as I use XP/2K3 anyway, but having been told my autovacuum service code needed to support NT4 I have been working on integrating pg_autovacuum into the backend, and I have it working, I'm just trying to clean up some lose ends before I submit another patch. I think backend integration will eliminate the need for your autovacuum service patch no? Matthew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PATCHES] stderr & win32 admin check
> -Original Message- > From: Tom Lane [mailto:[EMAIL PROTECTED] > Sent: 15 June 2004 14:58 > To: Magnus Hagander > Cc: Dave Page; [EMAIL PROTECTED] > Subject: Re: [PATCHES] stderr & win32 admin check > > "Magnus Hagander" <[EMAIL PROTECTED]> writes: > >> "Can't run Postgres securely" would be a > more-than-sufficient reason > >> not to support NT4, IMHO. > > > It could still be run on NT4 under the following conditions: > > 1) Running as a service > > 2) Running if the user logged in is not an administrator. > > Well, isn't "running as a service" sufficient? I thought > that was the only interesting case for non-hackers anyway. > > As long as you get an error message that's reasonably clear > about what you can do instead, this hardly seems like a showstopper... Well, that's kinda the point. If you are a hacker who has local admin privs (not exactly unusual on Windows networks - in some cases Power User group membership is required to run legacy software), you *cannot* run PostgreSQL except as a service, thus potentially making it a show stopper for those users. Personally I don't care as I use XP/2K3 anyway, but having been told my autovacuum service code needed to support NT4 Regards, Dave ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] stderr & win32 admin check
"Magnus Hagander" <[EMAIL PROTECTED]> writes: >> "Can't run Postgres securely" would be a more-than-sufficient >> reason not to support NT4, IMHO. > It could still be run on NT4 under the following conditions: > 1) Running as a service > 2) Running if the user logged in is not an administrator. Well, isn't "running as a service" sufficient? I thought that was the only interesting case for non-hackers anyway. As long as you get an error message that's reasonably clear about what you can do instead, this hardly seems like a showstopper... regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] stderr & win32 admin check
> > This will prevent PostgreSQL being runable on NT4 by anyone > with admin > > privileges, except as a service. > > Are we actually supporting NT4? I recall quite a bit of > discussion long ago about which versions of Windows were > really reasonable to support, but I don't recall if there was > a definite outcome. NT4 is on the list (for now). Nothing before it, and nothing from the 9x line. Meening NT4, 2000, XP and 2003 today. > "Can't run Postgres securely" would be a more-than-sufficient > reason not to support NT4, IMHO. It could still be run on NT4 under the following conditions: 1) Running as a service 2) Running if the user logged in is not an administrator. The difference is in 2000 and later you can log in with admin privs and then start postgres without it. //Magnus ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] stderr & win32 admin check
"Dave Page" <[EMAIL PROTECTED]> writes: > This will prevent PostgreSQL being runable on NT4 by anyone with admin > privileges, except as a service. Are we actually supporting NT4? I recall quite a bit of discussion long ago about which versions of Windows were really reasonable to support, but I don't recall if there was a definite outcome. "Can't run Postgres securely" would be a more-than-sufficient reason not to support NT4, IMHO. > I would suggest changing the error to a warning, or perhaps adding a > command line switch to 'downgrade' the error to a warning, thus forcing > an explicit action on the part of the user. See above. I don't want PG to become the next virus vector. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] stderr & win32 admin check
> -Original Message- > From: Magnus Hagander [mailto:[EMAIL PROTECTED] > Sent: 15 June 2004 09:16 > To: Dave Page; [EMAIL PROTECTED] > Subject: RE: [PATCHES] stderr & win32 admin check > > > > > > Oh, and I notice the use of the PowerUsers group - iirc, > there is no > > such group on NT4 domains, so the attempt to get the SID will fail. > > That is one weird NT4.. :-) > > First of all, "Power Users" is not a domain group, it is a > local group. > It has nothing to do with your domain. As such, it will also > work on non-domain-member servers/workstations. > > Second, the "Power Users" group have been included in the > local groups since NT 3.1, IIRC (could be 3.5). It is most > certainly there in NT4. Ack, you know I wrote that very late last night :-p You are of course correct that Power Users group is not a domain group. Wasn't aware that it was there in NT4 though - probably 'cos when I used to use NT4, it was on a small domain in which all servers were P/BDCs. Which don't have a power users group... /D ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] stderr & win32 admin check
> Oh, and I notice the use of the PowerUsers group - iirc, > there is no such group on NT4 domains, so the attempt to get > the SID will fail. That is one weird NT4.. :-) First of all, "Power Users" is not a domain group, it is a local group. It has nothing to do with your domain. As such, it will also work on non-domain-member servers/workstations. Second, the "Power Users" group have been included in the local groups since NT 3.1, IIRC (could be 3.5). It is most certainly there in NT4. This does not say anything about the other points you made - just this last one. //Magnus ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] stderr & win32 admin check
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of > Magnus Hagander > Sent: 14 June 2004 21:49 > To: [EMAIL PROTECTED] > Subject: [PATCHES] stderr & win32 admin check > > The patch makes the "if user is admin" check on win32 abort > startup, similar to unix. I know several people have said > they think this is wrong, since the last discussion. So if > someone wants to re-open that discussion, go ahead :-) OK, if you insist :-) This will prevent PostgreSQL being runable on NT4 by anyone with admin privileges, except as a service. NT4 doesn't have runas.exe (an su/sudo equivalent found on 2K/XP/2K3) thus preventing startup without logging in as a different user. I would guess that users with local admin privileges are far more common on Windows than users with root privileges on Unix systems, so this might affect more 'normal' users. Now I think of it, the default (and possibly only) user account on XP Home edition has admin privileges. I would suggest changing the error to a warning, or perhaps adding a command line switch to 'downgrade' the error to a warning, thus forcing an explicit action on the part of the user. Oh, and I notice the use of the PowerUsers group - iirc, there is no such group on NT4 domains, so the attempt to get the SID will fail. Regards, Dave. ---(end of broadcast)--- TIP 8: explain analyze is your friend
