Re: [HACKERS] PGXLOG variable worthwhile?
Hi everyone, In order to clarify things, how about we do a formal vote with specific details like this: *** Are you for... - pg_xlog directory changeable at all, not using symlinks? Yes/No - a PGXLOG environment variable to do this? Yes/No - a -X command line option to do this? Yes/No - a GUC (postgresql.conf) option to do this? Yes/No - altering the format of the pg_xlog directory so that it can't be used with the wrong database instance? Yes/No *** Does this seem reasonable? :-) Regards and best wishes, Justin Clift -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PGXLOG variable worthwhile?
On Thu, 26 Sep 2002, Jan Wieck wrote: > >>> But still, why set up a situation where your database might not > >>> start? Why not set it up so that if you get just *one* environment > >>> or command-line variable right, you can't set another inconsistently > >>> and screw up your start anyway? Why store configuration information > >>> outside of the database data directory in a form that's not easily > >>> backed up, and not easily found by other utilities? > > Apply that argumentation to all of our commandline switches and config > options and we end up with something that behaves like Microsoft > products ... they know everything better, you cannot tune them, they > work ... and you needed a bigger machine anyway. Talk about a straw man! I have repeatedly said: I WANT THE FEATURE THAT LETS YOU TUNE THE LOCATION OF THE LOG FILE! Read it again, and again, until you understand that we both want that feature. Then realize, I just want it implemented in a way that makes it less likely that people will find themselves in a situation where the server doesn't start. > I am absolutely not in favour of the PGXLOG environment variable. But if > someone else wants it, it doesn't bother me because I wouldn't use it > and it cannot hurt me. Responsible programmers, when confronted with a more accident-prone and less accident-prone way of doing something, chose the less accident-prone way of doing things. That way people who are naive, or tired, or just having a bad day are less likely to come to harm. Using the config file is not only safer, it's actually more convenient. And since we're going to have the config file option anyway, removing the environment variable option means that others have less documentation to read, and will spend less time wondering why there's two different ways to do the same thing. And naive people won't chose the wrong way because they don't know any better. cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
Curt Sampson wrote: > > On Wed, 25 Sep 2002, Jan Wieck wrote: > > > With the number of screws our product has, there are so many > > possible combinations that don't work, why worry about one more > > or less? > > That's just silly, so I won't even bother replying. Curt, it might sound silly on first sight and isolated. But it was in reply to: >>> But still, why set up a situation where your database might not >>> start? Why not set it up so that if you get just *one* environment >>> or command-line variable right, you can't set another inconsistently >>> and screw up your start anyway? Why store configuration information >>> outside of the database data directory in a form that's not easily >>> backed up, and not easily found by other utilities? Apply that argumentation to all of our commandline switches and config options and we end up with something that behaves like Microsoft products ... they know everything better, you cannot tune them, they work ... and you needed a bigger machine anyway. I am absolutely not in favour of the PGXLOG environment variable. But if someone else wants it, it doesn't bother me because I wouldn't use it and it cannot hurt me. I am simply against this "I think it's wrong so you have to change your behaviour" attitude. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PGXLOG variable worthwhile?
On Wed, 25 Sep 2002, Jan Wieck wrote: > With the number of screws our product has, there are so many > possible combinations that don't work, why worry about one more > or less? That's just silly, so I won't even bother replying. > Seriously, if you move around files, make symlinks or adjust > config variable to reflect that, there's allways the possibility > that you fatfinger it and cannot startup. True. But once your symlink is in place, it is stored on disk in the postgres data directory. An environment variable is a transient setting in memory, which means that you have to have a program set it, and you have to make sure that program gets run before any startup, be it an automated startup from /etc/rc on boot or a manual startup. > I want to have it it the config file. Well, then we're agreed. cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PGXLOG variable worthwhile?
Bruce Momjian <[EMAIL PROTECTED]> writes: > Can we wait for someone to be injured in a car accident before putting > in heavy seat belts? Not the analogy you wanted to make ... if you knew there was a serious risk, that's called negligence in most American courts. Ask Ford about the Pinto ... regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
OT: Seatbelts (was: Re: [HACKERS] PGXLOG variable worthwhile?)
Bruce Momjian wrote: > > scott.marlowe wrote: > > On Wed, 25 Sep 2002, Jan Wieck wrote: > > So, should we take out seatbelts from cars, safeties from guns, and have > > everyone run about with sharp sticks too? :-) I know that the second we > > make something more idiot proof, someone will make a better idiot, but > > that doesn't mean we shouldn't make things more idiot proof, we should > > just try to anticipate the majority of idiots (and let's face it, we can > > all be idiots at the right moments sometimes.) Sure, been there, done that ... > > Can we wait for someone to be injured in a car accident before putting > in heavy seat belts? About the car seatbelts I have a theory. If we would not have seatbelts, and instead of Airbags sharp sticks instantly killing the driver in the case of an accident, most of these wannabe Racing-Champs on our streets would either drive more reasonable or get removed by natural selection. Maybe the overall number of accidents would drop below the actual number of deaths in traffic (remember, we only kill the drivers on purpose, not anyone else in the car) ... and for sure the far lower number of *only* crippled or disabled victims will take a big burden off of the healthcare and wellfare system ... Okay, okay, enough proof of the first statement ... back to business. Jan B-) -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
"scott.marlowe" wrote: > [...] > But, I have a few more questions about the signature file solution. Is > the signature file going to be updated by date or something everytime the > database is started up and shut down? If not, then it's quite possible > that someone could copy the pg_xlog dir somewhere, run it for a while, > then they change it back to the base pg_xlog will the database know that > those xlogs are stale and not start up, or will it start up and corrupt > the database with the old xlogs? As long as there's a time stamp in both > places it should work fine. Good question. Actually, I think it'd be a perfect place and use for a copy of the controlfile. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
scott.marlowe wrote: > On Wed, 25 Sep 2002, Jan Wieck wrote: > So, should we take out seatbelts from cars, safeties from guns, and have > everyone run about with sharp sticks too? :-) I know that the second we > make something more idiot proof, someone will make a better idiot, but > that doesn't mean we shouldn't make things more idiot proof, we should > just try to anticipate the majority of idiots (and let's face it, we can > all be idiots at the right moments sometimes.) Can we wait for someone to be injured in a car accident before putting in heavy seat belts? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
On Wed, 25 Sep 2002, Jan Wieck wrote: > "scott.marlowe" wrote: > > > Having a FILE called pg_xlog isn't the fix here, it's the result of the > > fix, which is to take all the steps of moving the pg_xlog directory and > > put them into one script file the user doesn't need to understand to do it > > right. I.e. idiot proof the system as much as possible. > > And your script/program cannot modify postgresql.conf instead of > creating a new file? That's a minor point. It could be anywhere. It's just that much like a symlink is visible from the shell with a simple ls -l, so too is pg_xlog being a file an obvious sign that pg_xlog doesn't live here anymore. > Please remember: "A fool with a tool is still a fool". You can > provide programs and scripts as many as you want. There have > allways been these idiots who did stuff like truncating pg_log So, should we take out seatbelts from cars, safeties from guns, and have everyone run about with sharp sticks too? :-) I know that the second we make something more idiot proof, someone will make a better idiot, but that doesn't mean we shouldn't make things more idiot proof, we should just try to anticipate the majority of idiots (and let's face it, we can all be idiots at the right moments sometimes.) But, I have a few more questions about the signature file solution. Is the signature file going to be updated by date or something everytime the database is started up and shut down? If not, then it's quite possible that someone could copy the pg_xlog dir somewhere, run it for a while, then they change it back to the base pg_xlog will the database know that those xlogs are stale and not start up, or will it start up and corrupt the database with the old xlogs? As long as there's a time stamp in both places it should work fine. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PGXLOG variable worthwhile?
Jan Wieck wrote: > > It's almost like people *don't* want to put this in the config file > > or something > > I want to have it it the config file. Just that that doesn't > prevent anything. And if we have a "signature" file in the xlog > and data directories, you can make it dummy-safe as you like ... > if the config option is set wrong, first search for it on all > drives before bailing out and if found, postmaster corrects the > config setting. That way the admin can play hide and seek with > our database ... ;-) Let's get it into GUC and see what problems people have. We may find out no one has difficulty. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PGXLOG variable worthwhile?
"scott.marlowe" wrote: > Having a FILE called pg_xlog isn't the fix here, it's the result of the > fix, which is to take all the steps of moving the pg_xlog directory and > put them into one script file the user doesn't need to understand to do it > right. I.e. idiot proof the system as much as possible. And your script/program cannot modify postgresql.conf instead of creating a new file? Please remember: "A fool with a tool is still a fool". You can provide programs and scripts as many as you want. There have allways been these idiots who did stuff like truncating pg_log ... Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
Curt Sampson wrote: > > On Tue, 24 Sep 2002, Jan Wieck wrote: > > > And AFAICS it is scary only because screwing that up will simply corrupt > > your database. Thus, a simple random number (okay, and a timestamp of > > initdb) in two files, one in $PGDATA and one in $PGXLOG would be a > > totally sufficient safety mechanism to prevent starting with the wrong > > XLOG directory. > > But still, why set up a situation where your database might not > start? Why not set it up so that if you get just *one* environment > or command-line variable right, you can't set another inconsistently > and screw up your start anyway? Why store configuration information > outside of the database data directory in a form that's not easily > backed up, and not easily found by other utilities? With the number of screws our product has, there are so many possible combinations that don't work, why worry about one more or less? Seriously, if you move around files, make symlinks or adjust config variable to reflect that, there's allways the possibility that you fatfinger it and cannot startup. The point is not to make it pellethead-safe so that the damned thing will start allways, but to make it pellethead-safe so that an attempt to start with wrong settings doesn't blow away the whole server. > > It's almost like people *don't* want to put this in the config file > or something I want to have it it the config file. Just that that doesn't prevent anything. And if we have a "signature" file in the xlog and data directories, you can make it dummy-safe as you like ... if the config option is set wrong, first search for it on all drives before bailing out and if found, postmaster corrects the config setting. That way the admin can play hide and seek with our database ... ;-) Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
On Wed, 25 Sep 2002, Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > I don't see the gain of having a file called pg_xlog vs. using GUC. > > Well, the point is to have a safety interlock --- but I like Jan's > idea of using matching identification files in both directories. > With that, a GUC variable seems just fine. Agreed, the interlock is a great idea. I hadn't seen that one go by. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
Bruce Momjian <[EMAIL PROTECTED]> writes: > I don't see the gain of having a file called pg_xlog vs. using GUC. Well, the point is to have a safety interlock --- but I like Jan's idea of using matching identification files in both directories. With that, a GUC variable seems just fine. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
I do. The problem is that if you change the location of pg_xlog and do one thing wrong, poof, your database is now corrupt. Like Tom said earlier, imagine a command like switch called "please-dont-scram-my-database" and if you ever forgot it then your data is gone. Is it better to move such a switch into the postgresql.conf file? Imagine a GUC setting called "butter-and-bread" that when set would delete all your data. That's what the equivalent here is, if you make a single mistake. Having a FILE called pg_xlog isn't the fix here, it's the result of the fix, which is to take all the steps of moving the pg_xlog directory and put them into one script file the user doesn't need to understand to do it right. I.e. idiot proof the system as much as possible. We could do it much simpler, if everyone was on Unix. We could just write a script that would do everything the same but instead of using a file called pg_xlog, would make a link. the reason for the file is to make it more transportable to brain damaged OSes like Windows. Do you really think the GUC variable is a safe way of referencing the pg_xlog directory all by itself? I can see MANY posts to the lists that will go like this: I just installed Postgresql 7.4 and it's been working fine. I needed more speed, so I looked up the GUC for the pg_xlog and set it to /vol/vol3/ on my machine. Now my database won't come up. I set it back but it still won't come up. What can I do to fix that? Here's the email we'd get from my solution: Hey, I just tried to move my pg_xlog directory with the mvpgxlog script, and it gave an error of "permission denied on destination". What does that mean? The choice is yours. On Wed, 25 Sep 2002, Bruce Momjian wrote: > > I don't see the gain of having a file called pg_xlog vs. using GUC. > > --- > > scott.marlowe wrote: > > On Wed, 25 Sep 2002, Curt Sampson wrote: > > > > > On Tue, 24 Sep 2002, Jan Wieck wrote: > > > > > > > And AFAICS it is scary only because screwing that up will simply corrupt > > > > your database. Thus, a simple random number (okay, and a timestamp of > > > > initdb) in two files, one in $PGDATA and one in $PGXLOG would be a > > > > totally sufficient safety mechanism to prevent starting with the wrong > > > > XLOG directory. > > > > > > But still, why set up a situation where your database might not > > > start? Why not set it up so that if you get just *one* environment > > > or command-line variable right, you can't set another inconsistently > > > and screw up your start anyway? Why store configuration information > > > outside of the database data directory in a form that's not easily > > > backed up, and not easily found by other utilities? > > > > > > It's almost like people *don't* want to put this in the config file > > > or something > > > > Curt, did you see my post about this earlier? I'll repeat it now, just in > > case anyone else missed it. > > > > Problem: > > - People need to move the pg_xlog directory around on heavily > > loaded systems to improve performance > > > > Constraints: > > - Windows can't reliably use links to do this. > > - If the pg_xlog directory is moved wrong or referenced incorrectly, data > > corruption may occur. This makes using a switch or environmental var > > dangerous > > > > I consider using a GUC in the postgresql.conf file to be better than any > > other option listed so far, but it is still a dangerous place for it to > > be. > > > > So, the way I think that would work best would be: > > > > If there's a directory called pg_xlog in the $PGDATA directory, then use > > that. > > > > If there's a file called pg_xlog in the $PGDATA directory, then it will > > contain the path to the real pg_xlog directory. > > > > If you want to move the pg_xlog directory, you called a custom script > > called "mvpgxlog" or something like it that: > > > > 1: Checks to make sure the database is shut down > > 2: Checks to make sure the destination path has enough free space for the > > xlogs > > 3: If these are both true (and whatever logic we need here for safety) > > then copy the current pg_xlog directory contents to the new pg_xlog (even > > if we are already using an alternative location, this should work), set > > proper permissions, rename / move the pg_xlog file / directorry, then > > edit/create the $PGDATA/pg_xlog file to point to the new directory. > > > > This method has several advantages, and no real disadvantages I can think > > of. The advantages are: > > > > - It makes it easy to move the pg_xlog directory. > > - It works equally well for Windows and Unix. > > - Gets rid of another GUC setting people can scram their database with. > > - It is easy to backup your pg_xlog setting. > > - If painted green it should not rust. > > > > How's that sound for a general theory of operation? > > > > > > ---
Re: [HACKERS] PGXLOG variable worthwhile?
I don't see the gain of having a file called pg_xlog vs. using GUC. --- scott.marlowe wrote: > On Wed, 25 Sep 2002, Curt Sampson wrote: > > > On Tue, 24 Sep 2002, Jan Wieck wrote: > > > > > And AFAICS it is scary only because screwing that up will simply corrupt > > > your database. Thus, a simple random number (okay, and a timestamp of > > > initdb) in two files, one in $PGDATA and one in $PGXLOG would be a > > > totally sufficient safety mechanism to prevent starting with the wrong > > > XLOG directory. > > > > But still, why set up a situation where your database might not > > start? Why not set it up so that if you get just *one* environment > > or command-line variable right, you can't set another inconsistently > > and screw up your start anyway? Why store configuration information > > outside of the database data directory in a form that's not easily > > backed up, and not easily found by other utilities? > > > > It's almost like people *don't* want to put this in the config file > > or something > > Curt, did you see my post about this earlier? I'll repeat it now, just in > case anyone else missed it. > > Problem: > - People need to move the pg_xlog directory around on heavily > loaded systems to improve performance > > Constraints: > - Windows can't reliably use links to do this. > - If the pg_xlog directory is moved wrong or referenced incorrectly, data > corruption may occur. This makes using a switch or environmental var > dangerous > > I consider using a GUC in the postgresql.conf file to be better than any > other option listed so far, but it is still a dangerous place for it to > be. > > So, the way I think that would work best would be: > > If there's a directory called pg_xlog in the $PGDATA directory, then use > that. > > If there's a file called pg_xlog in the $PGDATA directory, then it will > contain the path to the real pg_xlog directory. > > If you want to move the pg_xlog directory, you called a custom script > called "mvpgxlog" or something like it that: > > 1: Checks to make sure the database is shut down > 2: Checks to make sure the destination path has enough free space for the > xlogs > 3: If these are both true (and whatever logic we need here for safety) > then copy the current pg_xlog directory contents to the new pg_xlog (even > if we are already using an alternative location, this should work), set > proper permissions, rename / move the pg_xlog file / directorry, then > edit/create the $PGDATA/pg_xlog file to point to the new directory. > > This method has several advantages, and no real disadvantages I can think > of. The advantages are: > > - It makes it easy to move the pg_xlog directory. > - It works equally well for Windows and Unix. > - Gets rid of another GUC setting people can scram their database with. > - It is easy to backup your pg_xlog setting. > - If painted green it should not rust. > > How's that sound for a general theory of operation? > > > ---(end of broadcast)--- > TIP 4: Don't 'kill -9' the postmaster > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
On Wed, 25 Sep 2002, Curt Sampson wrote: > On Tue, 24 Sep 2002, Jan Wieck wrote: > > > And AFAICS it is scary only because screwing that up will simply corrupt > > your database. Thus, a simple random number (okay, and a timestamp of > > initdb) in two files, one in $PGDATA and one in $PGXLOG would be a > > totally sufficient safety mechanism to prevent starting with the wrong > > XLOG directory. > > But still, why set up a situation where your database might not > start? Why not set it up so that if you get just *one* environment > or command-line variable right, you can't set another inconsistently > and screw up your start anyway? Why store configuration information > outside of the database data directory in a form that's not easily > backed up, and not easily found by other utilities? > > It's almost like people *don't* want to put this in the config file > or something Curt, did you see my post about this earlier? I'll repeat it now, just in case anyone else missed it. Problem: - People need to move the pg_xlog directory around on heavily loaded systems to improve performance Constraints: - Windows can't reliably use links to do this. - If the pg_xlog directory is moved wrong or referenced incorrectly, data corruption may occur. This makes using a switch or environmental var dangerous I consider using a GUC in the postgresql.conf file to be better than any other option listed so far, but it is still a dangerous place for it to be. So, the way I think that would work best would be: If there's a directory called pg_xlog in the $PGDATA directory, then use that. If there's a file called pg_xlog in the $PGDATA directory, then it will contain the path to the real pg_xlog directory. If you want to move the pg_xlog directory, you called a custom script called "mvpgxlog" or something like it that: 1: Checks to make sure the database is shut down 2: Checks to make sure the destination path has enough free space for the xlogs 3: If these are both true (and whatever logic we need here for safety) then copy the current pg_xlog directory contents to the new pg_xlog (even if we are already using an alternative location, this should work), set proper permissions, rename / move the pg_xlog file / directorry, then edit/create the $PGDATA/pg_xlog file to point to the new directory. This method has several advantages, and no real disadvantages I can think of. The advantages are: - It makes it easy to move the pg_xlog directory. - It works equally well for Windows and Unix. - Gets rid of another GUC setting people can scram their database with. - It is easy to backup your pg_xlog setting. - If painted green it should not rust. How's that sound for a general theory of operation? ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PGXLOG variable worthwhile?
On Tue, 24 Sep 2002, Jan Wieck wrote: > And AFAICS it is scary only because screwing that up will simply corrupt > your database. Thus, a simple random number (okay, and a timestamp of > initdb) in two files, one in $PGDATA and one in $PGXLOG would be a > totally sufficient safety mechanism to prevent starting with the wrong > XLOG directory. But still, why set up a situation where your database might not start? Why not set it up so that if you get just *one* environment or command-line variable right, you can't set another inconsistently and screw up your start anyway? Why store configuration information outside of the database data directory in a form that's not easily backed up, and not easily found by other utilities? It's almost like people *don't* want to put this in the config file or something cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
Jan Wieck <[EMAIL PROTECTED]> writes: > And AFAICS it is scary only because screwing that up will simply corrupt > your database. Thus, a simple random number (okay, and a timestamp of > initdb) in two files, one in $PGDATA and one in $PGXLOG would be a > totally sufficient safety mechanism to prevent starting with the wrong > XLOG directory. > Can we get that instead of ripping out anything? Sure, if someone wants to do that it'd go a long way towards addressing the safety issues. But given that, I think a GUC variable is the most appropriate control mechanism; as someone else pointed out, we've worked long and hard to make GUC useful and feature-ful, so it seems silly to invent new configuration items that bypass GUC. The safety concerns were the main reason I liked a symlink or separate file, but if we attack the safety problem directly then we might as well go for convenience in how you actually set the configuration value. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
Jan Wieck wrote: > Tom Lane wrote: > > > > "Marc G. Fournier" <[EMAIL PROTECTED]> writes: > > > You don't :vote: on stuff like this ... > > > > Why not, exactly? > > > > I wasn't aware that any of core had a non-vetoable right to apply > > any patch we liked regardless of the number and strength of the > > objections. AFAIK, we resolve differences of opinion by discussion, > > followed by a vote if the discussion doesn't produce a consensus. > > > > It was pretty clear that Thomas' original patch lost the vote, or > > would have lost if we'd bothered to hold a formal vote. I don't > > see anyone arguing against the notion of making XLOG location more > > easily configurable --- it was just the notion of making it depend > > on environment variables that scared people. > > And AFAICS it is scary only because screwing that up will simply corrupt > your database. Thus, a simple random number (okay, and a timestamp of > initdb) in two files, one in $PGDATA and one in $PGXLOG would be a > totally sufficient safety mechanism to prevent starting with the wrong > XLOG directory. > > Can we get that instead of ripping out anything? Well, the problem is that Thomas stopped communicating, perhaps because some were too aggressive in criticizing the patch. Once that happened, there was no way to come up with a solution, and that's why it was removed. Also, we are in the process of removing args and moving them to GUC so I don't see why we would make WAL an exception. It isn't changed that often. FYI, I am about to do the same removal for the SSL stuff too. Bear is no longer responding. It is on the open items list now. If I can't find someone who can review the good/bad parts of our SSL changes, it might all be yanked out. --- P O S T G R E S Q L 7 . 3 O P E NI T E M S Current at ftp://momjian.postgresql.org/pub/postgresql/open_items. Source Code Changes --- Schema handling - ready? interfaces? client apps? Drop column handling - ready for all clients, apps? Fix BeOS, QNX4 ports Fix AIX large file compile failure of 2002-09-11 (Andreas) Get bison upgrade on postgresql.org for ecpg only (Marc) Fix vacuum btree bug (Tom) Fix client apps for autocommit = off Fix clusterdb to be schema-aware Change log_min_error_statement to be off by default (Gavin) Fix return tuple counts/oid/tag for rules Loading 7.2 pg_dumps functions no longer public executable languages no longer public usable Add schema dump option to pg_dump Make SET not start a transaction with autocommit off, document it Add GRANT EXECUTE to all /contrib functions Revert or fix SSL change On Going Security audit Documentation Changes - Document need to add permissions to loaded functions and languages Move documation to gborg for moved projects -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
Tom Lane wrote: > > "Marc G. Fournier" <[EMAIL PROTECTED]> writes: > > You don't :vote: on stuff like this ... > > Why not, exactly? > > I wasn't aware that any of core had a non-vetoable right to apply > any patch we liked regardless of the number and strength of the > objections. AFAIK, we resolve differences of opinion by discussion, > followed by a vote if the discussion doesn't produce a consensus. > > It was pretty clear that Thomas' original patch lost the vote, or > would have lost if we'd bothered to hold a formal vote. I don't > see anyone arguing against the notion of making XLOG location more > easily configurable --- it was just the notion of making it depend > on environment variables that scared people. And AFAICS it is scary only because screwing that up will simply corrupt your database. Thus, a simple random number (okay, and a timestamp of initdb) in two files, one in $PGDATA and one in $PGXLOG would be a totally sufficient safety mechanism to prevent starting with the wrong XLOG directory. Can we get that instead of ripping out anything? Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
On Sun, 22 Sep 2002, Marc G. Fournier wrote: > Thomas implemented an option that he felt was useful, and that doesn't > break anything inside of the code ... he provided 2 methods of being able > to move the xlog's to another location (through command line and > environment variable, both of which are standard methods for doing such in > server software) ... but, because a small number of ppl "voted" that it > should go away, it went away ... The option as he implemented it did make the system more fragile. You can't back up an environment variable, it's separated from other configuration information, and it's more easily changed without realizing it. We should be building systems that are as resilient to human failure as possible, not opening up more possibilities of failure. We already have a place for configuration information: the configuration file. If I created a patch to move a variable out of the configuration file and make it an environment variable instead, everybody would (rightly) think I was nuts, and the patch certainly would not be accepted. So why should the situation be different for new configuration information? cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
"Nigel J. Andrews" wrote: > > And it's obvious it was centred on the use of an environment variable from the > subject line, it's still got PGXLOG in capitals in it. Actually, to be really precise, my original email asked for an environment variable. But only because I'd thought about it from the point of view of us already having a PGDATA environment variable and hadn't considered alternatives nor seen Thomas's stuff. Personally, I don't care if it's a -X, or an environment variable, or a GUC option. I'm just extremely positive that we should have an alternative to using symlinks for this (they don't work properly on NT). After following the discussion for a while I'm inclined to think that we should indeed have the GUC version, and *maybe* have the environment variable or the -X. The only thing bad about the -X is it's ability to trash your data if you forget it or get it wrong, and it's really easy to do in a decent scale environment with many servers. Marc has already suggested we might as well have something about a particular pg_xlog directory that PostgreSQL can use to check it's validity upon startup, so that could solve the data damaging issue. So, this thread has migrated away from a PGXLOG environment variable to discuss PGXLOG in general (good or bad) and also has implementation points too (about which people have been arguing). Regards and best wishes, Justin Clift > -- > Nigel J. Andrews > > ---(end of broadcast)--- > TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PGXLOG variable worthwhile?
On Sun, 22 Sep 2002, Tom Lane wrote: > > It was pretty clear that Thomas' original patch lost the vote, or > would have lost if we'd bothered to hold a formal vote. Hasn't there just been a formal vote on this? > I don't > see anyone arguing against the notion of making XLOG location more > easily configurable --- it was just the notion of making it depend > on environment variables that scared people. And it's obvious it was centred on the use of an environment variable from the subject line, it's still got PGXLOG in capitals in it. -- Nigel J. Andrews ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
"Marc G. Fournier" <[EMAIL PROTECTED]> writes: > You don't :vote: on stuff like this ... Why not, exactly? I wasn't aware that any of core had a non-vetoable right to apply any patch we liked regardless of the number and strength of the objections. AFAIK, we resolve differences of opinion by discussion, followed by a vote if the discussion doesn't produce a consensus. It was pretty clear that Thomas' original patch lost the vote, or would have lost if we'd bothered to hold a formal vote. I don't see anyone arguing against the notion of making XLOG location more easily configurable --- it was just the notion of making it depend on environment variables that scared people. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
Marc G. Fournier wrote: > > However, if you don't accept voting as a valid way to determine if a > > patch is acceptible, what method do you suggest? I don't think we want > > to go down the road of saying that you can't vote "no" on a feature > > addition. > > > > We just rejected a patch today on LIMIT with UPDATE/DELETE via an > > informal vote, and I think it was a valid rejection. > > Its not the concept of 'the vote', its what is being voted on that I have > a major problem with ... for instance, with the above LIMIT patch ... you > are talking about functionality ... I haven't seen that thread yet, so am > not sure why it was rejected, but did the submitter agree with the > reasons? Assuming he did, is this something he's going to re-submit later > after makign fixes? > > See, that is one thing I have enjoyed over the years ... someone submit's > a patch and a few ppl jump on top of it, point out a few problems iwth it > and the submitter re-submits with appropriate fixes ... > > Actually, I just went to my -patches folder and read the thread ... first > off, the 'informal vote' appears to have consisted of Tom Lane and Alvaro > Herrera, which isn't a vote ... second of all, in that case, the > implementation of such, I believe, would go against SQL specs, no? Second > of all, doesn't it just purely go against the point of a RDBMS if there > are multiple rows in a table with nothing to identify them except for the > ctid/oid? *scratch head* > > My point is, the use of an ENVIRONMENT variable for pointing ot a > directory is nowhere near on the scale of implementing an SQL statement > (or extension) that serves to take us steps backwards against the progress > we've made to improve our compliance ... The issue isn't really compliance because LIMIT in SELECT isn't compliant either, so adding it to UPDATE/DELETE is just as non-standard as in SELECT. The real question we vote on, I think, is, "Should this feature be given to our users? What value does it provide, and what confusion does it cause? Does the standard suggest anything?" I think that is the usual criteria. For LIMIT on UPDATE/DELETE, it provides little value, and adds confusion, i.e. an extra clause in those two commands that really doesn't add any functionality. Now, for the PG_XLOG environment variable/-X flag, it is almost the same result, i.e. it doesn't add much value (use a symlink) and does add confusion (oops, I forgot to set it). The idea of having the pg_xlog location in GUC I think was a good compromise, but too late to be discovered. If the patch author had continued discussion at the time, I think it would be in 7.3. > one has been removed due to personal preferences and nothign else ... the > other rejected as it will break (unless I've misread things?) standard, > accepted procedures ... PG_XLOG was remove for a few reasons: It didn't add much functionality It was ugly to add -X to all those commands It was error-prone Again, the same criteria. Are you saying the criteria I mentioned above is wrong? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
"Marc G. Fournier" <[EMAIL PROTECTED]> writes: > Ya know, I'm sitting back and reading this, and other threads, and > assimilating what is being bantered about, and start to think that > its time to cut back on the gatekeepers ... On the contrary, the quality of code accepted into a DBMS is really important. If you disagree with the definition of "code quality" that some developers are employing, then we can discuss that -- but I think that as the project matures, we should be more picky about the features we implement, not less. > Thomas implemented an option that he felt was useful, and that > doesn't break anything inside of the code The problem with this line of thinking is that "it doesn't break stuff" is not sufficient reason for adding a new feature. The burden of proof is on the person implementing the new feature. > ... he provided 2 methods of being able to move the xlog's to > another location Yes, but why do we need 2 different ways to do exactly the same thing? > but, because a small number of ppl "voted" that it should go away, > it went away ... They didn't just vote, they provided reasons why they thought the feature was brain-damaged -- reasons which have not be persuasively refuted, IMHO. If you'd like to see this feature in the code, might I suggest that you spend less time complaining about "gate keepers" (hint: it's called code review), and more time explaining exactly why the feature is worth having? Cheers, Neil -- Neil Conway <[EMAIL PROTECTED]> || PGP Key ID: DB3C29FC ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
On Sun, 22 Sep 2002, Bruce Momjian wrote: > Marc G. Fournier wrote: > > > > On Fri, 20 Sep 2002, Bruce Momjian wrote: > > > > > In fact, I tried to open a dialog with you on this issue several times, > > > but when I got no reply, I had to remove PGXLOG. If we had continued > > > discussion, we might have come up with the GUC compromise. > > > > Ya know, I'm sitting back and reading this, and other threads, and > > assimilating what is being bantered about, and start to think that its > > time to cut back on the gatekeepers ... > > > > Thomas implemented an option that he felt was useful, and that doesn't > > break anything inside of the code ... he provided 2 methods of being able > > to move the xlog's to another location (through command line and > > environment variable, both of which are standard methods for doing such in > > server software) ... but, because a small number of ppl "voted" that it > > should go away, it went away ... > > > > You don't :vote: on stuff like this ... if you don't like it, you just > > don't use it ... nobody is forcing you to do so. If you think there are > > going to be idiots out here that aren't going to use it right, then you > > document it appropriately, with *strong* wording against using it ... > > I understand your thought of reevaluating how we decide things. > > However, if you don't accept voting as a valid way to determine if a > patch is acceptible, what method do you suggest? I don't think we want > to go down the road of saying that you can't vote "no" on a feature > addition. > > We just rejected a patch today on LIMIT with UPDATE/DELETE via an > informal vote, and I think it was a valid rejection. Its not the concept of 'the vote', its what is being voted on that I have a major problem with ... for instance, with the above LIMIT patch ... you are talking about functionality ... I haven't seen that thread yet, so am not sure why it was rejected, but did the submitter agree with the reasons? Assuming he did, is this something he's going to re-submit later after makign fixes? See, that is one thing I have enjoyed over the years ... someone submit's a patch and a few ppl jump on top of it, point out a few problems iwth it and the submitter re-submits with appropriate fixes ... Actually, I just went to my -patches folder and read the thread ... first off, the 'informal vote' appears to have consisted of Tom Lane and Alvaro Herrera, which isn't a vote ... second of all, in that case, the implementation of such, I believe, would go against SQL specs, no? Second of all, doesn't it just purely go against the point of a RDBMS if there are multiple rows in a table with nothing to identify them except for the ctid/oid? *scratch head* My point is, the use of an ENVIRONMENT variable for pointing ot a directory is nowhere near on the scale of implementing an SQL statement (or extension) that serves to take us steps backwards against the progress we've made to improve our compliance ... one has been removed due to personal preferences and nothign else ... the other rejected as it will break (unless I've misread things?) standard, accepted procedures ... ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
Marc G. Fournier wrote: > > On Fri, 20 Sep 2002, Bruce Momjian wrote: > > > In fact, I tried to open a dialog with you on this issue several times, > > but when I got no reply, I had to remove PGXLOG. If we had continued > > discussion, we might have come up with the GUC compromise. > > Ya know, I'm sitting back and reading this, and other threads, and > assimilating what is being bantered about, and start to think that its > time to cut back on the gatekeepers ... > > Thomas implemented an option that he felt was useful, and that doesn't > break anything inside of the code ... he provided 2 methods of being able > to move the xlog's to another location (through command line and > environment variable, both of which are standard methods for doing such in > server software) ... but, because a small number of ppl "voted" that it > should go away, it went away ... > > You don't :vote: on stuff like this ... if you don't like it, you just > don't use it ... nobody is forcing you to do so. If you think there are > going to be idiots out here that aren't going to use it right, then you > document it appropriately, with *strong* wording against using it ... I understand your thought of reevaluating how we decide things. However, if you don't accept voting as a valid way to determine if a patch is acceptible, what method do you suggest? I don't think we want to go down the road of saying that you can't vote "no" on a feature addition. We just rejected a patch today on LIMIT with UPDATE/DELETE via an informal vote, and I think it was a valid rejection. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
On Fri, 20 Sep 2002, Bruce Momjian wrote: > In fact, I tried to open a dialog with you on this issue several times, > but when I got no reply, I had to remove PGXLOG. If we had continued > discussion, we might have come up with the GUC compromise. Ya know, I'm sitting back and reading this, and other threads, and assimilating what is being bantered about, and start to think that its time to cut back on the gatekeepers ... Thomas implemented an option that he felt was useful, and that doesn't break anything inside of the code ... he provided 2 methods of being able to move the xlog's to another location (through command line and environment variable, both of which are standard methods for doing such in server software) ... but, because a small number of ppl "voted" that it should go away, it went away ... You don't :vote: on stuff like this ... if you don't like it, you just don't use it ... nobody is forcing you to do so. If you think there are going to be idiots out here that aren't going to use it right, then you document it appropriately, with *strong* wording against using it ... ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
On Fri, 20 Sep 2002, Thomas Lockhart wrote: > Well, what I was hoping for, but no longer expect, is that features > (store xlog in another area) can be implemented and applied without > rejection by the new gatekeepers. It can be, and very simply. So long as you do it in the way which is not error-prone, rather than the way which is. > I have no fundamental objection to extending and replacing > implementation features as positive contributions to development. I do > have trouble with folks rejecting features without understanding the > issues, and sorry, there was a strong thread of "why would anyone want > to put storage on another device" to the discussion. I doubt it. There was perhaps a strong thread of "windows users are loosers," but certainly Unix folks put storage on another device all the time, using symlinks. This was mentioned many, many times. > There has been a fundamental shift in the quality and civility of > discussions over issues over the last couple of years, and I was naively > hoping that we could work through that on this topic. Not happening, and > not likely too. Well, when you're going to bring in Windows in a pretty heavily open-source-oriented group, no, it's not likely you're going to bring everyone together. (This is not a value judgement, it's just a, "Hello, this is the usenet (or something similar)," observation. That said, again, I don't think anybody was objecting to what you wanted to do. It was simply a bad implementation that I, and probably all the others, were objecting to. So please don't go on like we didn't like the concept. cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PGXLOG variable worthwhile?
Thomas Lockhart wrote: > ... > > Why you object to that, and insist it must be an environment variable > > instead (if that is indeed what you're doing), I'm not sure > > Well, what I was hoping for, but no longer expect, is that features > (store xlog in another area) can be implemented and applied without > rejection by the new gatekeepers. It is a feature that we do not have > now, and could have implemented for 7.3. > > No need to rehash the points which were not understood in the > "discussion". > > I have no fundamental objection to extending and replacing > implementation features as positive contributions to development. I do > have trouble with folks rejecting features without understanding the > issues, and sorry, there was a strong thread of "why would anyone want > to put storage on another device" to the discussion. I believe the discussion was "Why not use symlinks?" I think we have addressed that issue with the GUC variable solution. Certainly we all recognize the value of moving storage to another drive. It is mentioned in the SGML docs and other places. In fact, I tried to open a dialog with you on this issue several times, but when I got no reply, I had to remove PGXLOG. If we had continued discussion, we might have come up with the GUC compromise. > There has been a fundamental shift in the quality and civility of > discussions over issues over the last couple of years, and I was naively > hoping that we could work through that on this topic. Not happening, and > not likely too. My impression is that things have been getting better in the past six months. There is more open discussion, and more voting, meaning one group isn't making all the decisions. I have worked to limit the sway of any "new gatekeepers". People are encouraged to vote, and we normally accept that outcome. I think gatekeepers should sway only in the force of their arguments. Do you feel this was not followed on the PGXLOG case, or is the concept in error? I certainly have been frustrated when my features were not accepted, but I have to accept the vote of the group. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PGXLOG variable worthwhile?
On Thu, 19 Sep 2002, Thomas Lockhart wrote: > Actually, a core member did implement this just a few weeks ago. The > same crew arguing this time rejected the changes and removed them from > the 7.3 feature set. The change to make a PG_XLOG environment variable was rejected. Is that really the change you were talking about? > So some folks have their heels dug in, and the vocal ones are not really > interested in understanding the issues which this feature is addressing. I was one of the vocal objectors, and I certainly understand the issues very well. Perhaps we should be saying the vocal supporters of the environment variable don't understand the issues. None of the objectors I saw have any problem with enabling Windows NT to have the log file somewhere else. In fact, I'm very strongly in support of this. But I object to doing it in a way that makes the system more fragile and susceptable to not starting properly, or even damage, when there's a simple and obvious way of doing it right: put this in the database configuration file rather than in an environment variable. Why you object to that, and insist it must be an environment variable instead (if that is indeed what you're doing), I'm not sure cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
Robert Treat wrote: > It seems all of this discussion misses the point. Either it has a large > amount of impact and the idea gets rejected because of implementation > issues, or it has little impact but it's nothing the core group wants to > implement. If the problem is finding someone to implement it, it sounds > like Justin has found such a person, so are we going to stand in his way > while we wax poetic about OS religion and corporate philosophies or can > he start submitting patches? Well, I have Win32 patches here I am reviewing. I think I can say that the changes are minimal and probably will be accepted for addition into 7.4. I am actually surprised at how little is required. Right now, 7.4 is targeted with point-in-time recovery and Win32. And, in fact, both patches are almost ready for inclusion into CVS, so we may find that 7.4 has a very short release cycle. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
It seems all of this discussion misses the point. Either it has a large amount of impact and the idea gets rejected because of implementation issues, or it has little impact but it's nothing the core group wants to implement. If the problem is finding someone to implement it, it sounds like Justin has found such a person, so are we going to stand in his way while we wax poetic about OS religion and corporate philosophies or can he start submitting patches? Robert Treat On Mon, 2002-09-16 at 14:11, Justin Clift wrote: > Peter Eisentraut wrote: > > > > Justin Clift writes: > > > > > WinNT/2k/XP costs a few hundred dollars. > > > > > > MS SQL Server costs a few thousand dollars. > > > > The places that run Windows can be categorized into three camps: > > > How about this? > > The places that run Windows can be categorised a number of different > ways, depending on what you're looking for. > > 1) Places that have in-house staff that can do or learn everything. > > Many of these places are really small, some are not. PostgreSQL fits > well here, Windows or not, as these people are prepared to learn how to > use it best. > > > 2) Companies that hire external IT services. > > Often the software implemented here will be dependent on outside sources > of advice such as consultants, executives who take an interest in IT > mags, etc. > > Look at Windows NT on the server in the first place. Microsoft > leveraged the marketplace through making itself available then promoting > the heck out of itself into the IT press, industry mags, etc. > > These places will be receptive to PostgreSQL as our reputation further > becomes known and they can see where PostgreSQL will be useful to them. > PostgreSQL on Win NT/2K/XP will definitely be of use to a sizable number > of these businesses. > > > 3) Companies who depend on multiple external sources of IT support. > i.e. One reasonable sized enterprise here in Australia has over 450 > *development* companies presently working on applications for their > environment. Because of the scope of standardisation needed, they > standardised on WinNT many years ago. It still works for them. They > don't even have SP6 installed on their desktops as it breaks too many of > the desktop applications. etc. > > These people are not clueless. They make strategic decisions when > they're necessary, and it all comes down to flexibility, reliability, > and cost. > > For some things they run Unix, or Windows, or Novell, or OS/390, or any > number of other stuff. > > Because of the years of experience some of their support companies have > with WinNT, it works reliably enough for them. They don't have the > "need to reboot once per week" thing with their servers. > > These guys will become receptive to PostgreSQL too, and it will be in > our favour to be able to demonstrate very good performance across all > platforms that we can, not just our own *personally preferred* > platforms. > > By giving them options when it doesn't take a *whole bunch of effort* to > do so, we open up ways for PostgreSQL to be used that we haven't even > thought of before. We all know this already. > > It wouldn't really surprise me greatly if at some point this proved > beneficial to a non-Windows platform for some reason too. > > :-) > > Regards and best wishes, > > Justin Clift > > > > So where is the market? > > > > -- > > Peter Eisentraut [EMAIL PROTECTED] > > -- > "My grandfather once told me that there are two kinds of people: those > who work and those who take the credit. He told me to try to be in the > first group; there was less competition there." >- Indira Gandhi > > ---(end of broadcast)--- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to [EMAIL PROTECTED] so that your > message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
Rod Taylor wrote: > I forget, is it possible to make a GUC that cannot be changed during > runtime? Yes, you can set it to it only can be changed by the super-user and only takes effect on restart. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
I forget, is it possible to make a GUC that cannot be changed during runtime? If so, then I vote yes, otherwise, there is a problem if someone tries. On Tue, 2002-09-17 at 17:07, Bruce Momjian wrote: > Dave Page wrote: > > Which in this case is what puzzles me. We are only talking about a > > simple GUC variable after all - I don't know for sure, but I'm guessing > > it's not a huge effort to add one? > > Can we get agreement on that? A GUC for pg_xlog location? Much cleaner > than -X, doesn't have the problems of possible accidental use, and does > allow pg_xlog moving without symlinks, which some people don't like? -- Rod Taylor ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
Nigel J. Andrews wrote: > On Tue, 17 Sep 2002, Bruce Momjian wrote: > > > Dave Page wrote: > > > Which in this case is what puzzles me. We are only talking about a > > > simple GUC variable after all - I don't know for sure, but I'm guessing > > > it's not a huge effort to add one? > > > > Can we get agreement on that? A GUC for pg_xlog location? Much cleaner > > than -X, doesn't have the problems of possible accidental use, and does > > allow pg_xlog moving without symlinks, which some people don't like? > > > > If I can get a few 'yes' votes I will add it to TODO and do it for 7.4. > > GUC instead of -X or PGXLOG : yes. > > However, how is that going to work if tablespaces are introduced in 7.4. Surely > the same mechanism for tablespaces would be used for pg_xlog. As the tablespace > mechanism hasn't been determined yet, as far as I know, wouldn't it be best to > see what happens there before creating the TODO item for the log? Good point. How about: Allow pg_xlog to be moved without symlinks That is vague enough. Added to TODO. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
"Nigel J. Andrews" wrote: > However, how is that going to work if tablespaces are introduced in 7.4. Surely > the same mechanism for tablespaces would be used for pg_xlog. As the tablespace > mechanism hasn't been determined yet, as far as I know, wouldn't it be best to > see what happens there before creating the TODO item for the log? It's a Yes from me of course. Would a TODO list entry of something like "Add a GUC xlog_path variable" be broad enough that people keep it in mind when tablespaces are created, but it doesn't get forgotten about by not being on the list? :-) Regards and best wishes, Justin Clift > -- > Nigel J. Andrews > Director > > --- > Logictree Systems Limited > Computer Consultants > > ---(end of broadcast)--- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PGXLOG variable worthwhile?
On Tue, 17 Sep 2002, Bruce Momjian wrote: > Dave Page wrote: > > Which in this case is what puzzles me. We are only talking about a > > simple GUC variable after all - I don't know for sure, but I'm guessing > > it's not a huge effort to add one? > > Can we get agreement on that? A GUC for pg_xlog location? Much cleaner > than -X, doesn't have the problems of possible accidental use, and does > allow pg_xlog moving without symlinks, which some people don't like? > > If I can get a few 'yes' votes I will add it to TODO and do it for 7.4. GUC instead of -X or PGXLOG : yes. However, how is that going to work if tablespaces are introduced in 7.4. Surely the same mechanism for tablespaces would be used for pg_xlog. As the tablespace mechanism hasn't been determined yet, as far as I know, wouldn't it be best to see what happens there before creating the TODO item for the log? -- Nigel J. Andrews Director --- Logictree Systems Limited Computer Consultants ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
Dave Page wrote: > Which in this case is what puzzles me. We are only talking about a > simple GUC variable after all - I don't know for sure, but I'm guessing > it's not a huge effort to add one? Can we get agreement on that? A GUC for pg_xlog location? Much cleaner than -X, doesn't have the problems of possible accidental use, and does allow pg_xlog moving without symlinks, which some people don't like? If I can get a few 'yes' votes I will add it to TODO and do it for 7.4. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
> -Original Message- > From: Christopher Kings-Lynne [mailto:[EMAIL PROTECTED]] > Sent: 17 September 2002 09:49 > To: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Cc: [EMAIL PROTECTED] > Subject: Re: [HACKERS] PGXLOG variable worthwhile? > > > Users HAVE provided their feedback - they want Postgres on > Windows. What's the point of open source if we can't > accomodate them? There's no problems with economics, > marketing, schedules, deadlines, nothing. The reason that > people like Open Source is because they don't have to deal > with some monolithic company refusing to port to their > platform just because it's "too hard". Which in this case is what puzzles me. We are only talking about a simple GUC variable after all - I don't know for sure, but I'm guessing it's not a huge effort to add one? Regards, Dave. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
> Let's leave it. The main point to focus postgres on unix is not > only because > unix is proven/known as robust and scalable, but unix is much > more standard to > support across multiple OS. The amount with which windows differs > from unices > on API level, any serious efforts to make postgresql good enough > on windows > whould be a mammoth task. It's already been done - that's the whole point. > So if users are not providing their feedback, what's the point in > open source? Users HAVE provided their feedback - they want Postgres on Windows. What's the point of open source if we can't accomodate them? There's no problems with economics, marketing, schedules, deadlines, nothing. The reason that people like Open Source is because they don't have to deal with some monolithic company refusing to port to their platform just because it's "too hard". Chris ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PGXLOG variable worthwhile?
> -Original Message- > From: Shridhar Daithankar > [mailto:[EMAIL PROTECTED]] > Sent: 17 September 2002 09:30 > To: [EMAIL PROTECTED] > Cc: [EMAIL PROTECTED] > Subject: Re: [HACKERS] PGXLOG variable worthwhile? > > > On 17 Sep 2002 at 16:11, Christopher Kings-Lynne wrote: > > But I disagree. History says that nobody can compete with > microsoft on > microsoft platform. Postgres will not be competing with > either SQL Server or > access. It would remain as toy database.. Like Oracle? > Let's leave it. The main point to focus postgres on unix is > not only because > unix is proven/known as robust and scalable, but unix is much > more standard to > support across multiple OS. The amount with which windows > differs from unices > on API level, any serious efforts to make postgresql good > enough on windows > whould be a mammoth task. Maybe, but it's pretty much there now. The beta Win32 native port has been performing excellently in the tests I've been able to throw at it, certainly better than the Cygwin port. > I haven't tried either port of postgres on windows but I > would not bet on any > of them. The thing I wouldn't bet on is not the quality of the code produced by the developers here, but Windows. Yes, it runs great here at the moment, and has done for a while now but there's no guarantee that a new release won't have a nasty bug. But that applies to the SQL user as well though. Or for that matter the user of *any* other OS... > There are limits as what postgresql can do on windows and > probably postgresql > development team can't do much about many of them.. The only real issue afaik with the current beta is that you can only run one instance on a single server. That is the case with SQL Server as well of course. > No offense to anybody.. just some opinions.. > Likewise. Regards, Dave. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
> -Original Message- > From: Christopher Kings-Lynne [mailto:[EMAIL PROTECTED]] > Sent: 17 September 2002 09:05 > To: Dave Page; Bruce Momjian > Cc: Robert Treat; Justin Clift; Peter Eisentraut; Tom Lane; > Curt Sampson; PostgreSQL Hackers Mailing List > Subject: RE: [HACKERS] PGXLOG variable worthwhile? > > > > I use PostgreSQL on Linux for production and XP for > development, and > > am likely to continue that way. I've been beta testing the native > > Win32 port of PostgreSQL as Justin has and the latest version is > > fantastic - it runs as a service, osdb shows impressive results > > compared to Cygwin PostgreSQL on the same system and it's a > breeze to > > install, despite there being no installer yet. > > >From where do we get this fabled Win32 port? > The call for testers (below) was originally posted to the Cygwin list. Regards, Dave. === My company is actively working on a Native Windows Port of Postgres based on 7.2.1. This is the same group that Jan Wieck and Katie Ward work with. We are now at the stage that we need community involvement to help work out the bugs. We plan on contributing the code to the Postgres base, but we want to make sure that most of the bugs have been worked out before doing so. We are looking for people who have an application that currently runs on Postgres 7.2 and who also have a Windows environment. If you would like to get involved, please send me email at mailto:[EMAIL PROTECTED] Thanks... ...MikeF -- -- Mike Furgal - mailto:[EMAIL PROTECTED] - http://www.multera.com -- ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
On 17 Sep 2002 at 16:11, Christopher Kings-Lynne wrote: > > What I can't understand is the attitude of some people here. Yes, > > Microsoft are evil, but the bottom line is, millions of people use > > Windows. Just look at the number of downloads for pgAdmin (shown at > > http://www.pgadmin.org/downloads/) - the last stable version has clocked > > up over 38,000 downloads, the preview I released just a couple of weeks > > ago, 2230 at the time of writing. I know from talking to some of the > > users that often people download copies for themselves and their > > colleagues, so we can probably assume there are actually 40,000+ > > PostgreSQL users that use Windows reguarly enough to want pgAdmin. What > > happens if you add in the pgAccess/Windows users, Tora, or pgExplorer? > > How many of these people would want to run PostgreSQL on Windows as > > well? > I actually think that the long-term survival of Postgres DEPENDS on our > Win32 support. Otherwise, we'll just get massacred by MySQL, MSSQL, Oracle > and Firebird who do support Win32. Let's move this to general. But I disagree. History says that nobody can compete with microsoft on microsoft platform. Postgres will not be competing with either SQL Server or access. It would remain as toy database.. As far as people using mysql on windows, I have couple of colleages here who got things crowling for some heavy load, something like 60GB database with 512MB compq workstations.. Let's leave it. The main point to focus postgres on unix is not only because unix is proven/known as robust and scalable, but unix is much more standard to support across multiple OS. The amount with which windows differs from unices on API level, any serious efforts to make postgresql good enough on windows whould be a mammoth task. I haven't tried either port of postgres on windows but I would not bet on any of them. > Users of Postgres are our lifeblood. The more users we have the more I agree but even as of now, not even 1% users comes on any of postgres lists, in my estimate. So if users are not providing their feedback, what's the point in open source? (Actually all those people do help postgres by publicising it but still feedback remains an important phase of open source software engineering..) > developers we get, the more testing we get and the more likely we are to get > money, corporate support, etc. Our ODBC driver will also be improved. I agree for ODBC but that can be done without giving much to postgresql windows port as well. I understand windows port of postgresql remains very much important for people who want to evaluate it. But for some good evaluation, I would rather recommend them trying postgresql on linux rather than windows. There are limits as what postgresql can do on windows and probably postgresql development team can't do much about many of them.. No offense to anybody.. just some opinions.. Bye Shridhar -- Albrecht's Law: Social innovations tend to the level of minimum tolerable well- being. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
> What I can't understand is the attitude of some people here. Yes, > Microsoft are evil, but the bottom line is, millions of people use > Windows. Just look at the number of downloads for pgAdmin (shown at > http://www.pgadmin.org/downloads/) - the last stable version has clocked > up over 38,000 downloads, the preview I released just a couple of weeks > ago, 2230 at the time of writing. I know from talking to some of the > users that often people download copies for themselves and their > colleagues, so we can probably assume there are actually 40,000+ > PostgreSQL users that use Windows reguarly enough to want pgAdmin. What > happens if you add in the pgAccess/Windows users, Tora, or pgExplorer? > How many of these people would want to run PostgreSQL on Windows as > well? I actually think that the long-term survival of Postgres DEPENDS on our Win32 support. Otherwise, we'll just get massacred by MySQL, MSSQL, Oracle and Firebird who do support Win32. Users of Postgres are our lifeblood. The more users we have the more developers we get, the more testing we get and the more likely we are to get money, corporate support, etc. Our ODBC driver will also be improved. Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
> I use PostgreSQL on Linux for production and XP for development, and am > likely to continue that way. I've been beta testing the native Win32 > port of PostgreSQL as Justin has and the latest version is fantastic - > it runs as a service, osdb shows impressive results compared to Cygwin > PostgreSQL on the same system and it's a breeze to install, despite > there being no installer yet. >From where do we get this fabled Win32 port? Chris ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
> -Original Message- > From: Bruce Momjian [mailto:[EMAIL PROTECTED]] > Sent: 17 September 2002 06:36 > To: Christopher Kings-Lynne > Cc: Robert Treat; Justin Clift; Peter Eisentraut; Tom Lane; > Curt Sampson; PostgreSQL Hackers Mailing List > Subject: Re: [HACKERS] PGXLOG variable worthwhile? > > > Well, let's see if we ever run on native NT4.X and we can > decide then. > Actually, don't our Cygnus folks have a problem with moving > pg_xlog already? No, because Cygwin knows about shell links. Whilst I'm here, I'll chuck my $0.02 in: I use PostgreSQL on Linux for production and XP for development, and am likely to continue that way. I've been beta testing the native Win32 port of PostgreSQL as Justin has and the latest version is fantastic - it runs as a service, osdb shows impressive results compared to Cygwin PostgreSQL on the same system and it's a breeze to install, despite there being no installer yet. What I can't understand is the attitude of some people here. Yes, Microsoft are evil, but the bottom line is, millions of people use Windows. Just look at the number of downloads for pgAdmin (shown at http://www.pgadmin.org/downloads/) - the last stable version has clocked up over 38,000 downloads, the preview I released just a couple of weeks ago, 2230 at the time of writing. I know from talking to some of the users that often people download copies for themselves and their colleagues, so we can probably assume there are actually 40,000+ PostgreSQL users that use Windows reguarly enough to want pgAdmin. What happens if you add in the pgAccess/Windows users, Tora, or pgExplorer? How many of these people would want to run PostgreSQL on Windows as well? What about the companies out there that have good sysadmins who want to use PostgreSQL, but manglement that insist on using Windows? What about situations where a single server is running SQL Server and other software (such as a middle tier server - as I have on one box here), and that other software cannot be changed, but SQL could? I think that ignoring the huge number of people that use windows because some of us consider it a Mickey Mouse OS is a particuarly bad strategy if we want to expand our userbase. Windows is not going anywhere soon, and like it or not, it *is* getting better and better. Our Windows 2000 (and our Beta3/RC1 .Net test Servers) are rock solid and haven't been rebooted in months) - we get more hardware faults these days, and those can occur on our Linux or HP-UX boxes just as easily. Anyway, enough of my rant :-) Regards, Dave. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
Christopher Kings-Lynne wrote: > > > It seems all of this discussion misses the point. Either it has a large > > > amount of impact and the idea gets rejected because of implementation > > > issues, or it has little impact but it's nothing the core group wants to > > > implement. If the problem is finding someone to implement it, it sounds > > > like Justin has found such a person, so are we going to stand in his way > > > while we wax poetic about OS religion and corporate philosophies or can > > > he start submitting patches? > > > > Actually, the work is minimal. Look at the commit I used to remove > > PGXLOG, trim that to remove the changes to make the path name dynamic in > > size (added too much complexity for little benefit) and hang the path > > coding off a GUC variable rather than an environment variable. > > I personally don't see the problem with a GUC variable...that seems like the > perfect solution to me... Well, let's see if we ever run on native NT4.X and we can decide then. Actually, don't our Cygnus folks have a problem with moving pg_xlog already? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
> > It seems all of this discussion misses the point. Either it has a large > > amount of impact and the idea gets rejected because of implementation > > issues, or it has little impact but it's nothing the core group wants to > > implement. If the problem is finding someone to implement it, it sounds > > like Justin has found such a person, so are we going to stand in his way > > while we wax poetic about OS religion and corporate philosophies or can > > he start submitting patches? > > Actually, the work is minimal. Look at the commit I used to remove > PGXLOG, trim that to remove the changes to make the path name dynamic in > size (added too much complexity for little benefit) and hang the path > coding off a GUC variable rather than an environment variable. I personally don't see the problem with a GUC variable...that seems like the perfect solution to me... Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PGXLOG variable worthwhile?
Robert Treat wrote: > It seems all of this discussion misses the point. Either it has a large > amount of impact and the idea gets rejected because of implementation > issues, or it has little impact but it's nothing the core group wants to > implement. If the problem is finding someone to implement it, it sounds > like Justin has found such a person, so are we going to stand in his way > while we wax poetic about OS religion and corporate philosophies or can > he start submitting patches? Actually, the work is minimal. Look at the commit I used to remove PGXLOG, trim that to remove the changes to make the path name dynamic in size (added too much complexity for little benefit) and hang the path coding off a GUC variable rather than an environment variable. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
Curt Sampson <[EMAIL PROTECTED]> writes: > On Thu, 12 Sep 2002, Justin Clift wrote: >> Am just wondering if we've ever considered adding a PGXLOG environment >> variable that would point to the pg_xlog directory? > IMHO, a much better way to support this is to put this information into > the config file. That way it can't easily change when you happen to, say, > start postgres in the wrong window. Yes. We rejected environment-variable-based xlog location for reasons that apply equally well to Windows. The xlog location *must* be stored in a physical file in the data directory; anything else is too unsafe. The current technology for that is a symlink. While it doesn't have to be a symlink as opposed to some sort of config file, I don't have the slightest problem with saying that we don't support relocation of xlog on older Windoid platforms. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
I wrote: > scott.marlowe wrote: >> >> I wouldn't assume that. It's been years since I tested it, but back >> then, the command line and all program I used could see the link >> created by ln that came with the resource kit. They were distinctly >> different from the shortcut type of links, in that they seems >> transparent like short cuts in unix generally are. >> >> Do you have the resource kit or the gnu utils from it? > > > The situation appears to be this: > > 1. Soft links are available on NTFS 5 (2K/XP) as Reparse Points via the > DeviceIoControl() function for any application using the standard C > library routines. > > 2. Soft links are available on any filesystem under 95/98/ME/NT4/2K/XP > as OLE streams (.lnk files) for Shell-aware applications. > > 3. Hard links are available on NTFS 5 (2K/XP) via the CreateHardLink() API. > 4. Hard links are available on NTFS (NT3.1/NT4) via the BackupWrite() > API by writing a special stream to the NTFS. I also believe (I could be wrong) that for directories, the only two methods of links are the Soft link methods above. So PGXLOG cannot use soft links on a non-XP/2K machine unless it is "Shell-Aware". For example, in a cygwin bash command window: mkdir dir1 ln dir1 dir2 <- Error using Cygwin implementation ln -s dir1 dir2 <- Creates a Shell short-cut (NT4) echo "Hello" > dir1/test.txt cat dir2/test.txt "Hello" <- Cygwin's cat(bash?) is shell short-cut aware Now, in a Windows NT command prompt: notepad dir2\test.txt <- Notepad can't find file notepad dir2.lnk <- Displays link contents That means for a native port with a different PGXLOG directory running on NT4, the only choice *using links* is to make the native port shell short-cut aware. I could be wrong but I don't think so. Mike Mascari [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
scott.marlowe wrote: > On Fri, 13 Sep 2002, Justin Clift wrote: > >>Would it be correct to say that the 'ln' command in the MS Resource Kit >>creates this kind of shortcut too, as the Reparse Points feature doesn't >>seem to be possible under NT4? > > > I wouldn't assume that. It's been years since I tested it, but back then, > the command line and all program I used could see the link created by ln > that came with the resource kit. They were distinctly different from the > shortcut type of links, in that they seems transparent like short cuts in > unix generally are. > > Do you have the resource kit or the gnu utils from it? The situation appears to be this: 1. Soft links are available on NTFS 5 (2K/XP) as Reparse Points via the DeviceIoControl() function for any application using the standard C library routines. 2. Soft links are available on any filesystem under 95/98/ME/NT4/2K/XP as OLE streams (.lnk files) for Shell-aware applications. 3. Hard links are available on NTFS 5 (2K/XP) via the CreateHardLink() API. See: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/createhardlink.asp 4. Hard links are available on NTFS (NT3.1/NT4) via the BackupWrite() API by writing a special stream to the NTFS. Example: http://www.mvps.org/win32/ntfs/lnw.cpp The cygwin implementation of link(): http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/winsup/cygwin/syscalls.cc?rev=1.149.2.23&content-type=text/x-cvsweb-markup&cvsroot=src 1. Will use CreateHardLink() if on 2K/XP 2. Will try to use the BackupWrite() method 3. Failing #2 will just copy the file See how fun Microsoft makes things? Mike Mascari [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PGXLOG variable worthwhile?
On Thu, 12 Sep 2002, Justin Clift wrote: > Am just wondering if we've ever considered adding a PGXLOG environment > variable that would point to the pg_xlog directory? IMHO, a much better way to support this is to put this information into the config file. That way it can't easily change when you happen to, say, start postgres in the wrong window. cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PGXLOG variable worthwhile?
On Fri, 13 Sep 2002, Justin Clift wrote: > "scott.marlowe" wrote: > > > > Seems like the NT4 users are left out in the cold though until we add > > > some kind of ability for PostgreSQL to not look at the filesystem for > > > info about where to put the xlog files. > > > > This isn't true. With the resource kit, you get the gnu utils, and ln > > works a charm under NT4 with ntfs. And not just for directories, but > > files as well. Unless Microsoft somehow removed that functionality in the > > intervening years since I've used NT. (wouldn't put it past them, but I > > doubt they have.) > > The reference point that I'm working from is this: > > - Am testing out the third beta of the Native PostgreSQL port for > Windows, on NT4 SP6 at present. > - Have an internal RAID array of Seagate Cheetah 10kRPM drives. When > installing the PGDATA directory on one drive it gives a certain kind of > performance, and I'm interested in testing the performance of the Native > PostgreSQL port for Windows with the xlog directory being located on > another drive. > - Have tried doing normal shortcuts, and have also tried using the > cygwin "ln" command to create the appropriate soft link. Both > approaches create a shortcut object of the correct name pointing to the > correct place on the new drive, but the only thing that appears to > follow this shortcut is when I click on them using Windows Explorer. > The Native PostgreSQL port for Windows doesn't, and neither do a few > other applications I tested. > > Would it be correct to say that the 'ln' command in the MS Resource Kit > creates this kind of shortcut too, as the Reparse Points feature doesn't > seem to be possible under NT4? I wouldn't assume that. It's been years since I tested it, but back then, the command line and all program I used could see the link created by ln that came with the resource kit. They were distinctly different from the shortcut type of links, in that they seems transparent like short cuts in unix generally are. Do you have the resource kit or the gnu utils from it? Looking at this url: http://unxutils.sourceforge.net/ the part for ln.exe says it makes real hard links on ntfs (which means they would be on the same drive.) So I'm not sure if ntfs supports soft links across volumes transparently or not now. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PGXLOG variable worthwhile?
scott.marlowe wrote: > > Seems like the NT4 users are left out in the cold though until we add > > some kind of ability for PostgreSQL to not look at the filesystem for > > info about where to put the xlog files. > > This isn't true. With the resource kit, you get the gnu utils, and ln > works a charm under NT4 with ntfs. And not just for directories, but > files as well. Unless Microsoft somehow removed that functionality in the > intervening years since I've used NT. (wouldn't put it past them, but I > doubt they have.) Yes, this is what I remember, that Cygwin had symlinks, and at that time that was the only Win32 OS we supported. Now, with native Win32 port coming, we have to figure out what is available. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PGXLOG variable worthwhile?
"scott.marlowe" wrote: > > Seems like the NT4 users are left out in the cold though until we add > > some kind of ability for PostgreSQL to not look at the filesystem for > > info about where to put the xlog files. > > This isn't true. With the resource kit, you get the gnu utils, and ln > works a charm under NT4 with ntfs. And not just for directories, but > files as well. Unless Microsoft somehow removed that functionality in the > intervening years since I've used NT. (wouldn't put it past them, but I > doubt they have.) The reference point that I'm working from is this: - Am testing out the third beta of the Native PostgreSQL port for Windows, on NT4 SP6 at present. - Have an internal RAID array of Seagate Cheetah 10kRPM drives. When installing the PGDATA directory on one drive it gives a certain kind of performance, and I'm interested in testing the performance of the Native PostgreSQL port for Windows with the xlog directory being located on another drive. - Have tried doing normal shortcuts, and have also tried using the cygwin "ln" command to create the appropriate soft link. Both approaches create a shortcut object of the correct name pointing to the correct place on the new drive, but the only thing that appears to follow this shortcut is when I click on them using Windows Explorer. The Native PostgreSQL port for Windows doesn't, and neither do a few other applications I tested. Would it be correct to say that the 'ln' command in the MS Resource Kit creates this kind of shortcut too, as the Reparse Points feature doesn't seem to be possible under NT4? Can only think of two real solutions at present, one being for us to add a PGXLOG environment variable or similar ability (GUC parameter perhaps?), and the other would be for the Native PostgreSQL for Windows port to follow these shortcuts. Not if any of these is all that easy, or maybe there is another solution that would work (apart from ignoring the problem). :-) Regards and best wishes, Justin Clift -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] PGXLOG variable worthwhile?
On Thu, 12 Sep 2002, Justin Clift wrote: > Mike Mascari wrote: > > > In Windows 2000 and Windows XP with an NTFS filesystem, > > Microsoft has added Reparse Points, which allow for the > > implementation of symbolic links for directories. Microsoft > > calls them "Junctions". I *believe* the function used for > > creating reparse points is DeviceIoControl() with the > > FSCTL_SET_REPARSE_POINT I/O control code. I don't have quick > > access to 2K or XP, but it is clearly not supported by Win32 on > > 95/98/ME. > > > > Here's a link discussing the features of NTFS5 and Reparse Points: > > > > >http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnw2kmag00/html/NTFSPart1.asp > > That's really useful info. Reparse points under Win2k (mount points to > the rest of us) are definitely something to try out in the future then. > :) > > Seems like the NT4 users are left out in the cold though until we add > some kind of ability for PostgreSQL to not look at the filesystem for > info about where to put the xlog files. This isn't true. With the resource kit, you get the gnu utils, and ln works a charm under NT4 with ntfs. And not just for directories, but files as well. Unless Microsoft somehow removed that functionality in the intervening years since I've used NT. (wouldn't put it past them, but I doubt they have.) ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
Mike Mascari wrote: > In Windows 2000 and Windows XP with an NTFS filesystem, > Microsoft has added Reparse Points, which allow for the > implementation of symbolic links for directories. Microsoft > calls them "Junctions". I *believe* the function used for > creating reparse points is DeviceIoControl() with the > FSCTL_SET_REPARSE_POINT I/O control code. I don't have quick > access to 2K or XP, but it is clearly not supported by Win32 on > 95/98/ME. > > Here's a link discussing the features of NTFS5 and Reparse Points: > > >http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnw2kmag00/html/NTFSPart1.asp That's really useful info. Reparse points under Win2k (mount points to the rest of us) are definitely something to try out in the future then. :) Seems like the NT4 users are left out in the cold though until we add some kind of ability for PostgreSQL to not look at the filesystem for info about where to put the xlog files. Regards and best wishes, Justin Clift > Mike Mascari > [EMAIL PROTECTED] -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PGXLOG variable worthwhile?
Dave Page wrote: > >>-Original Message- >>From: Bruce Momjian [mailto:[EMAIL PROTECTED]] >> >>Also, I have heard symlinks are available in native Windows >>but the interface to them isn't clearly visible. Can someone >>clarify that? > > > Well there are 'shortcuts' but I wouldn't want to trust my xlog > directory to one. These are Shell OLE links. As Dave points out, it requires the shell to interpret the shortcut. > > Even if I did, iirc, unless you are using the shell api, they just > appear to be regular files anyway (for example, in Cygwin vi, I can edit > a shortcut to a directory). > > Regards, Dave. In Windows 2000 and Windows XP with an NTFS filesystem, Microsoft has added Reparse Points, which allow for the implementation of symbolic links for directories. Microsoft calls them "Junctions". I *believe* the function used for creating reparse points is DeviceIoControl() with the FSCTL_SET_REPARSE_POINT I/O control code. I don't have quick access to 2K or XP, but it is clearly not supported by Win32 on 95/98/ME. Here's a link discussing the features of NTFS5 and Reparse Points: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnw2kmag00/html/NTFSPart1.asp Mike Mascari [EMAIL PROTECTED] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] PGXLOG variable worthwhile?
> -Original Message- > From: Bruce Momjian [mailto:[EMAIL PROTECTED]] > Sent: 12 September 2002 06:27 > To: Justin Clift > Cc: PostgreSQL Hackers Mailing List > Subject: Re: [HACKERS] PGXLOG variable worthwhile? > > Also, I have heard symlinks are available in native Windows > but the interface to them isn't clearly visible. Can someone > clarify that? Well there are 'shortcuts' but I wouldn't want to trust my xlog directory to one. Even if I did, iirc, unless you are using the shell api, they just appear to be regular files anyway (for example, in Cygwin vi, I can edit a shortcut to a directory). Regards, Dave. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] PGXLOG variable worthwhile?
We dealt this this (painfully) during 7.3 development. Some wanted a -X flag to initdb/postgres/postmaster that would identify the pg_xlog directory while others wanted the flag only on initdb and have initdb create a symlink. Finally, we decided to do nothing. and continue to recommend manually moving pg_xlog using symlinks. Also, I have heard symlinks are available in native Windows but the interface to them isn't clearly visible. Can someone clarify that? --- Justin Clift wrote: > Hi everyone, > > Am just wondering if we've ever considered adding a PGXLOG environment > variable that would point to the pg_xlog directory? > > In a Unix environment it's not real necessary as filesystem links can be > created, but in other environments (i.e. the Native windows port) it's > looking like it might be useful. > > :-) > > Regards and best wishes, > > Justin Clift > > -- > "My grandfather once told me that there are two kinds of people: those > who work and those who take the credit. He told me to try to be in the > first group; there was less competition there." >- Indira Gandhi > > ---(end of broadcast)--- > TIP 4: Don't 'kill -9' the postmaster > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
