Re: [HACKERS] Call for 7.5 feature completion
Marc G. Fournier wrote: Personally, I think there are alot of large features that ppl have been hard at getting complete in time for June 1st that we should stick to it, else we're going to end up with 'yet another release' delayed in hopes that the outstanding bugs in Win32 will get fixed in a reasonable amount of time ... June 1st, let's do beta for 7.5 and then branch onto 8.0, with 8.0 key'd to the Win32 Native port being finished ... If that means 8.0 happens to be September 1st, so be it ... Bruce agreed that this had been vague before today, so if people have had this date in mind and have been working to it, perhaps they have some telepathic abilities I lack ... We missed on PITR *and* Win32 last year. ISTM there's a bit of a credibility issue at stake, so it might well be worth taking a couple of weeks leeway if that's what is required. The other point, especially about Win32, is to see if we can spread the load a bit. Perhaps Claudio, Magnus, Merlin and Bruce should start trying to farm out specific tasks. I for one will be very upset if it misses this release. cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Call for 7.5 feature completion
Tom Lane wrote: Marc G. Fournier [EMAIL PROTECTED] writes: *If* June 1st comes along, and Win32 isn't ready, there is nothing wrong with freezing the code *except* for a pending Win32 patch ... Yeah there is ... In the first place it's unfair to other developers to make schedule slips at the last moment, and especially to *plan* to do so. I would prefer we determine at least two weeks, preferably a month, in advance whether there is going to be a schedule change. I think it's entirely appropriate to decide now whether we are going to hold to June 1 feature freeze or slip it. That gives other people a chance to do useful work while we approach the deadline. In the second, any Win32 issues that remain unresolved as of June 1 are likely to be major enough (in terms of code impact) that we can't realistically contemplate slipping in post-beta patches for them. The fallback position is going to have to be no (supported) Win32 port in 7.5, not we'll fix this in beta. Agreed. No last minute delay of the feature freeze, and no pushing major code in during beta. It sounds good now, but it makes for a big mess when it happens. We should decide now or mid-May if June 1 will be the feature freeze date. Marc is concerned that folks will not work hard to meet a deadline and will slack off if we push thing to July 1. However, looking at the open features, I haven't seen any slacking on these projects. Simon said he would submit PITR patches at the end of April, and he did. Alvaro started at the beginning of April and he is on schedule. Win32 has continued on a steady pace for six months now. What we might need to do during this time is to remind folks to submit patches for feedback regularly to keep the community involved in their progress. I think Marc's logic is that we are going to get X features into 7.5 whether we set a June 1 or July 1 feature cuttoff date. I don't think that. I would say we might get X if we choose June 1 and 1.5X if we go for July 1, based on the current state of these projects, and also based on the fact we didn't advertize the June 1 clearly enough months ago. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 29 Apr 2004, Bruce Momjian wrote: Marc is concerned that folks will not work hard to meet a deadline and will slack off if we push thing to July 1. I don't think I'm so muc worried about that as under-estimating the amount of time required to finish them ... its not like *that's* never happened before, and with something as big as some of the features that are in the works, and with some of the major show-stoppers that the Win32 port has brought up, I can seriously see '1 month' being drawn out further ... Win32 has continued on a steady pace for six months now. Be honest ... 6 months ago, did you believe the Win32 work would have taken 6 months? How many of the current issues could you have anticipated? How many will crop up in the next month? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] PITR logging control program
Simon Riggs wrote: Agreed we want to allow the superuser control over writing of the archive logs. The question is how do they get access to that. Is it by running a client program continuously or calling an interface script from the backend? My point was that having the backend call the program has improved reliablity and control over when to write, and easier administration. Agreed. We've both suggested ways that can occur, though I suggest this is much less of a priority, for now. Not no, just not now. Another case is server start/stop. You want to start/stop the archive logger to match the database server, particularly if you reboot the server. I know Informix used a client program for logging, and it was a pain to administer. pg_arch is just icing on top of the API. The API is the real deal here. I'm not bothered if pg_arch is not accepted, as long as we can adopt the API. As noted previously, my original mind was to split the API away from the pg_arch application to make it clearer what was what. Once that has been done, I encourage others to improve pg_arch - but also to use the API to interface with other BAR prodiucts. If you're using PostgreSQL for serious business then you will be using a serious BAR product as well. There are many FOSS alternatives... The API's purpose is to allow larger, pre-existing BAR products to know when and how to retrieve data from PostgreSQL. Those products don't and won't run underneath postmaster, so although I agree with Peter's original train of thought, I also agree with Tom's suggestion that we need an API more than we need an archiver process. I would be happy with an exteral program if it was started/stoped by the postmaster (or via GUC change) and received a signal when a WAL file was written. That is exactly what has been written. The PostgreSQL side of the API is written directly into the backend, in xlog.c and is therefore activated by postmaster controlled code. That then sends a signal to the process that will do the archiving - the Archiver side of the XLogArchive API has it as an in-process library. (The signal is, in fact, a zero-length file written to disk because there are many reasons why an external archiver may not be ready to archive or even up and running to receive a signal). The only difference is that there is some confusion as to the role and importance of pg_arch. OK, I have finalized my thinking on this. We both agree that a pg_arch client-side program certainly works for PITR logging. The big question in my mind is whether a client-side program is what we want to use long-term, and whether we want to release a 7.5 that uses it and then change it in 7.6 to something more integrated into the backend. Let me add this is a little different from pg_autovacuum. With that, you could put it in cron and be done with it. With pg_arch, there is a routine that has to be used to do PITR, and if we change the process in 7.6, I am afraid there will be confusion. Let me also add that I am not terribly worried about having the feature to restore to an arbitrary point in time for 7.5. I would much rather have a good PITR solution that works cleanly in 7.5 and add it to 7.6, than to have retore to an arbitrary point but have a strained implementation that we have to revisit for 7.6. Here are my ideas. (I talked to Tom about this and am including his ideas too.) Basically, the archiver that scans the xlog directory to identify files to be archived should be a subprocess of the postmaster. You already have that code and it can be moved into the backend. Here is my implementation idea. First, your pg_arch code runs in the backend and is started just like the statistics process. It has to be started whether PITR is being used or not, but will be inactive if PITR isn't enabled. This must be done because we can't have a backend start this process later in case they turn on PITR after server start. The process id of the archive process is stored in shared memory. When PITR is turned on, each backend that complete a WAL file sends a signal to the archiver process. The archiver wakes up on the signal and scans the directory, finds files that need archiving, and either does a 'cp' or runs a user-defined program (like scp) to transfer the file to the archive location. In GUC we add: pitr = true/false pitr_location = 'directory, [EMAIL PROTECTED]:/dir, etc' pitr_transfer = 'cp, scp, etc' The archiver program updates its config values when someone changes these values via postgresql.conf (and uses pg_ctl reload). These can only be modified from postgresql.conf. Changing them via SET has to be disabled because they are cluster-level settings, not per session, like port number or checkpoint_segments. Basically, I think that we need to push user-level control of this process down beyond the directory scanning code (that is pretty standard),
Re: [HACKERS] Call for 7.5 feature completion
The other point, especially about Win32, is to see if we can spread the load a bit. Perhaps Claudio, Magnus, Merlin and Bruce should start trying to farm out specific tasks. Here are the tasks, off the Win32 page, I see as necessary to drop in: * Handle sync() by opening all file opened since the last fsync and fsync'ing those - Tom's got this one, as is the most crucial outstanding part * changes to replace PGBINDIR, PGDATADIR (as per PKGLIBDIR) - I'm waiting for a reply from Tom/Peter for suggestions, as objectors to a previous patch * locatime under Win32 with pre-1970 dates - Magnus has submitted; Bruce will commit any time now * Win32 installer - I believe Magnus already has something in this regard * Win32 service - Rony offered to do this, but have not heard back. Not a major item, and I've already got an implementation from my previous port. * Fix problem were locale doesn't match host locale, requires passing more params to startup applications - SMOP All other issues are nicities or fixups, IMHO. There really isn't anything that needs farming out, just a matter of having the pieces fall into place. I for one will be very upset if it misses this release. If I have anything to do with it, this won't happen. I'll do whatever I can to make the release. That's my 2c. I don't see anything that can't make a June 1 deadline (assuming we are expected to keep to it! :-)... the only unknown for me is the sync/fsync code Tom is doing, only as I have no idea where he is up to. I've been AWOL for a month, but will again start actively contributing, but as things stand there isn't a great deal left to do to take Win32 to beta. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Call for 7.5 feature completion
Marc G. Fournier wrote: On Thu, 29 Apr 2004, Bruce Momjian wrote: Marc is concerned that folks will not work hard to meet a deadline and will slack off if we push thing to July 1. I don't think I'm so muc worried about that as under-estimating the amount of time required to finish them ... its not like *that's* never happened before, and with something as big as some of the features that are in the works, and with some of the major show-stoppers that the Win32 port has brought up, I can seriously see '1 month' being drawn out further ... Win32 has continued on a steady pace for six months now. Be honest ... 6 months ago, did you believe the Win32 work would have taken 6 months? How many of the current issues could you have anticipated? How many will crop up in the next month? Honestly, I didn't think we would be this far by now (we had regression tests mostly passing 4-6 weeks ago), and the win32 status page hasn't added a new TODO item in quite some time. I thought we would be hacking Win32 all through 7.5 beta and releasing something after 7.5 as a 7.5 plus patches release but we are far enough along to hit 7.5 by feature freeze. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Call for 7.5 feature completion
Marc G. Fournier [EMAIL PROTECTED] writes: On Thu, 29 Apr 2004, Tom Lane wrote: In the first place it's unfair to other developers to make schedule slips at the last moment, and especially to *plan* to do so. Isn't it equally unfair to slip the scheduale that developers that have been working on some large features (PITR, 2PC immediately coming to mind) have been working towards based on a deadline? If Win32 that much more important then those other features? As you well know, I have no use for the Win32 port at all ;-). However, of the major features that Bruce just listed, the Win32 port is the only one I consider really likely to appear in 7.5; sure it needs major work yet, but the others are still in the vaporware-till-proven-otherwise category. Certainly they are not solid enough to justify making schedule decisions on the basis of this will probably be ready by date X. I am willing to adjust the freeze deadline now to make it more probable that at least one of those major features will really make it into 7.5. The realities are that the Win32 port should determine any such schedule decision, because nothing else is close enough to the finish line to justify considering its needs instead. I guess my point is really do you want to freeze on June 1 if *none* of these features are done? regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Call for 7.5 feature completion
Win32 has continued on a steady pace for six months now. Be honest ... 6 months ago, did you believe the Win32 work would have taken 6 months? How many of the current issues could you have anticipated? How many will crop up in the next month? FWIW, the backend porting effort started less than 5 months ago :-P What current issues are there? The only ones of any significance that popped up were the APC/socket interactions and the localtime deficiencies, both of which now appear solved. Perhaps I will regret saying this but from my point of view, apart from the fsync/sync code Tom is working on that I am in complete ignorance of, I do not imagine any new issues in the remaining items required to get win32 to beta. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see a href=http://www.memetrics.com/emailpolicy.html;http://www.memetrics.com/em ailpolicy.html/a ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Call for 7.5 feature completion
Claudio Natoli wrote: Win32 has continued on a steady pace for six months now. Be honest ... 6 months ago, did you believe the Win32 work would have taken 6 months? How many of the current issues could you have anticipated? How many will crop up in the next month? FWIW, the backend porting effort started less than 5 months ago :-P What current issues are there? The only ones of any significance that popped up were the APC/socket interactions and the localtime deficiencies, both of which now appear solved. Perhaps I will regret saying this but from my point of view, apart from the fsync/sync code Tom is working on that I am in complete ignorance of, I do not imagine any new issues in the remaining items required to get win32 to beta. Agreed. My major issue is that we don't have enough _big_ items completed to set a feature freeze date. I think we need X big items completed before we can push for 7.5, even forgetting Win32. I know we did a lot already, but we need some big splash items for 7.5. These are being worked on, and if it takes X months to get them, it takes X months. Once we have X features working, we can look for a feature freeze. Maybe those features will not even include Win32 (though I would be surprised). -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Call for 7.5 feature completion
Tom Lane wrote: Marc G. Fournier [EMAIL PROTECTED] writes: On Thu, 29 Apr 2004, Tom Lane wrote: In the first place it's unfair to other developers to make schedule slips at the last moment, and especially to *plan* to do so. Isn't it equally unfair to slip the scheduale that developers that have been working on some large features (PITR, 2PC immediately coming to mind) have been working towards based on a deadline? If Win32 that much more important then those other features? As you well know, I have no use for the Win32 port at all ;-). However, of the major features that Bruce just listed, the Win32 port is the only one I consider really likely to appear in 7.5; sure it needs major work yet, but the others are still in the vaporware-till-proven-otherwise category. Certainly they are not solid enough to justify making schedule decisions on the basis of this will probably be ready by date X. I am willing to adjust the freeze deadline now to make it more probable that at least one of those major features will really make it into 7.5. The realities are that the Win32 port should determine any such schedule decision, because nothing else is close enough to the finish line to justify considering its needs instead. I guess my point is really do you want to freeze on June 1 if *none* of these features are done? Yep, my point too, that we need X big features to schedule beta, and we don't have any yet. I believe PITR actually does work as Simon has tested it, and we have the code. Of course, i am discussing how it should be integrated, but I do believe it works. And I think Gavin will complete his tablespaces, perhaps with our help. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 29 Apr 2004, Tom Lane wrote: I guess my point is really do you want to freeze on June 1 if *none* of these features are done? No, I agree that that would be foolish ... but there has also been alot done on the code over the past few months that even *one* of those features should be enough to put it over the top ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Call for 7.5 feature completion
Marc G. Fournier wrote: On Thu, 29 Apr 2004, Tom Lane wrote: I guess my point is really do you want to freeze on June 1 if *none* of these features are done? No, I agree that that would be foolish ... but there has also been alot done on the code over the past few months that even *one* of those features should be enough to put it over the top ... But we should have at least one done before setting a freeze date, and the freeze date should be one month in the future. Right now we have none. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Call for 7.5 feature completion
Claudio Natoli [EMAIL PROTECTED] writes: That's my 2c. I don't see anything that can't make a June 1 deadline (assuming we are expected to keep to it! :-)... the only unknown for me is the sync/fsync code Tom is doing, only as I have no idea where he is up to. I've been AWOL for a month, but will again start actively contributing, but as things stand there isn't a great deal left to do to take Win32 to beta. I am willing to commit to making the fsync thing happen by June 1. If you are confident that everything else can be fixed by then, my vote is to hold to the June 1 schedule. (I was just panicking because some other folk didn't think everything else could get done...) regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Call for 7.5 feature completion
Marc G. Fournier wrote: On Thu, 29 Apr 2004, Matthew T. O'Connor wrote: I know it's a chicken and egg problem, do we set a date for developers to shoot for, or do shoot for specific features and choose a date from there. I think there can no hard and fast rule on this, it depends on the feature and the desire to put it in place. If we set the date based on a specific featuer, v7.4 would still be in development ... And if we always set deadlines independent of the required development time, then we may never get a win32 port or any other major feature that takes a little more time and attention. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Call for 7.5 feature completion
On Fri, 30 Apr 2004, Bruce Momjian wrote: Marc G. Fournier wrote: On Thu, 29 Apr 2004, Tom Lane wrote: I guess my point is really do you want to freeze on June 1 if *none* of these features are done? No, I agree that that would be foolish ... but there has also been alot done on the code over the past few months that even *one* of those features should be enough to put it over the top ... But we should have at least one done before setting a freeze date, and the freeze date should be one month in the future. Right now we have none. Now, that logic *doesn't* follow ... freeze date == that one feature makes more sense ... or freeze date == one month from May 1st *or* one feature, whichever is longer makes sense ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Call for 7.5 feature completion
Marc G. Fournier wrote: On Fri, 30 Apr 2004, Bruce Momjian wrote: Marc G. Fournier wrote: On Thu, 29 Apr 2004, Tom Lane wrote: I guess my point is really do you want to freeze on June 1 if *none* of these features are done? No, I agree that that would be foolish ... but there has also been alot done on the code over the past few months that even *one* of those features should be enough to put it over the top ... But we should have at least one done before setting a freeze date, and the freeze date should be one month in the future. Right now we have none. Now, that logic *doesn't* follow ... freeze date == that one feature makes more sense ... or freeze date == one month from May 1st *or* one feature, whichever is longer makes sense ... But we don't want to have all our developers controlled by one feature being completed. It isn't fair. They should get a good warning about freeze starting. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Call for 7.5 feature completion
On Fri, 30 Apr 2004, Matthew T. O'Connor wrote: And if we always set deadlines independent of the required development time, then we may never get a win32 port or any other major feature that takes a little more time and attention. Actually, that one doesn't hold ... it just means that we need to mature our development environments to match the larger projects that need to happen ... we are using CVS, there is nothing that stops a side-branch being started, like we did with the Win32 stuff and then merging it back up when the feature is ready .. Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Call for 7.5 feature completion
On Fri, 30 Apr 2004, Bruce Momjian wrote: But we don't want to have all our developers controlled by one feature being completed. It isn't fair. They should get a good warning about freeze starting. Nor is it fair to extend the development cycle indefinitely waiting for that one feature in the first place ... there is *nothing* that states that the 7.6 dev cycle is nice and short for a change ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Call for 7.5 feature completion
Marc G. Fournier wrote: On Fri, 30 Apr 2004, Bruce Momjian wrote: But we don't want to have all our developers controlled by one feature being completed. It isn't fair. They should get a good warning about freeze starting. Nor is it fair to extend the development cycle indefinitely waiting for that one feature in the first place ... there is *nothing* that states that the 7.6 dev cycle is nice and short for a change ... Agreed. I say we wait for _a_ major feature completed, or X major features. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Current CVS tip segfaulting
Alvaro Herrera [EMAIL PROTECTED] writes: In current (as of a couple hours ago) clean CVS tip sources, without any of my local changes, I'm getting a postmaster segfault when trying to connect to a non existant database. Alvaro, did you figure this out? I've been mostly distracted for the past week ... regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] How does postgres remain the unique index unique?
Hello,everyone, I want to know how postgres remain its unique index unique after i have change it,Forexample: CREATE TABLE human(age INT,name CHAR(30));then in pg_attribute catalog the unique index pg_attribute_relid_attnum_index will use the relid of'human' and attnum of age to create a index tuple for attribute:age,which is a unique index tuple describingage ,then i do ALTER TABLE human RENAMEage TO birthdate;in the unique index pg_attribute_relid_attnum_index will appear two index tuple which keys are relid of 'human'and attnum:1 to describe age(In my mind,when insert or update a heap tuple,postgres only inserts a new index tuple for it in its index relation if it has not delete or tag the old index tuple),but there is no errorhappeningSo i am interested in the method of remianing the dynamic index unique very much. would you please tell me how postgres completes it and the src of this?thank:) == 263 == 263
[HACKERS]
hello,everyone!I want to know how postgresql remain its unique index unique after i have change it.For example:create table human(age int,name char(2));then in the pg_attribute catalog the unique index pg_attribute_relid_attnum_index willuse relid of 'human'and attnum of age to create a index tuple for attribute:age,which is a unique index tuple for a tuple which describe age in pg_attribute,and i do:alter table human rename age to birthdate;then in the unique index pg_attribute_relid_attnum_index will appear two index tuple which keys are relid of 'human' and attnum:1 to describe age ,but there is no error happeningSo i am interested in the method of remain the dynamic index very much.would you please tell how postgres do it and the src of this? thanks :) [EMAIL PROTECTED]2004-04-29 == 263
Re: [HACKERS] PITR logging control program
On Thu, Apr 29, 2004 at 12:18:38AM -0400, Bruce Momjian wrote: OK, I looked over the code. Basically it appears pg_arch is a client-side program that copies files from pg_xlog to a specified directory, and marks completion in a new pg_rlog directory. The driving part of the program seems to be: while ( (n = read( xlogfd, buf, BLCKSZ)) 0) if ( write( archfd, buf, n) != n) return false; The program basically sleeps and when it awakes checks to see if new WAL files have been created. Is the API able to indicate a written but not-yet-filled WAL segment? So an archiver could copy the filled part, and refill it later. This may be needed because a segment could take a while to be filled. -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) Hoy es el primer día del resto de mi vida ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PITR logging control program
Alvaro Herrera wrote: On Thu, Apr 29, 2004 at 12:18:38AM -0400, Bruce Momjian wrote: OK, I looked over the code. Basically it appears pg_arch is a client-side program that copies files from pg_xlog to a specified directory, and marks completion in a new pg_rlog directory. The driving part of the program seems to be: while ( (n = read( xlogfd, buf, BLCKSZ)) 0) if ( write( archfd, buf, n) != n) return false; The program basically sleeps and when it awakes checks to see if new WAL files have been created. Is the API able to indicate a written but not-yet-filled WAL segment? So an archiver could copy the filled part, and refill it later. This may be needed because a segment could take a while to be filled. I couldn't figure that out, but I don't think it does. It would have to lock the WAL writes so it could get a good copy, I think, and I didn't see that. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PITR logging control program
On Thu, Apr 29, 2004 at 10:07:01AM -0400, Bruce Momjian wrote: Alvaro Herrera wrote: Is the API able to indicate a written but not-yet-filled WAL segment? So an archiver could copy the filled part, and refill it later. This may be needed because a segment could take a while to be filled. I couldn't figure that out, but I don't think it does. It would have to lock the WAL writes so it could get a good copy, I think, and I didn't see that. I'm not sure but I don't think so. You don't have to lock the WAL for writing, because it will always write later in the file than you are allowed to read. (If you read more than you were told to, it's your fault as an archiver.) -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) Et put se mouve (Galileo Galilei) ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PITR logging control program
Alvaro Herrera wrote: On Thu, Apr 29, 2004 at 10:07:01AM -0400, Bruce Momjian wrote: Alvaro Herrera wrote: Is the API able to indicate a written but not-yet-filled WAL segment? So an archiver could copy the filled part, and refill it later. This may be needed because a segment could take a while to be filled. I couldn't figure that out, but I don't think it does. It would have to lock the WAL writes so it could get a good copy, I think, and I didn't see that. I'm not sure but I don't think so. You don't have to lock the WAL for writing, because it will always write later in the file than you are allowed to read. (If you read more than you were told to, it's your fault as an archiver.) My point was that without locking the WAL, we might get part of a WAL write in our file, but I now realize that during a crash the same thing might happen, so it would be OK to just copy it even if it is being written to. Simon posted the rest of his patch that shows changes to the backend, and a comment reads: + * The name of the notification file is the message that will be picked up + * by the archiver, e.g. we write RLogDir/000100C6.full + * and the archiver then knows to archive XLOgDir/000100C6, + * while it is doing so it will rename RLogDir/000100C6.full + * to RLogDir/000100C6.busy, then when complete, rename it again + * to RLogDir/000100C6.done so it is only archiving full logs. Also, I think this archiver should be able to log to a local drive, network drive (trivial), tape drive, ftp, or use an external script to transfer the logs somewhere. (ftp would probably be an external script with 'expect'). -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] 7.5 features
Tom Lane said: Bruce Momjian [EMAIL PROTECTED] writes: What we really need is for these folks to start finalizing their patches and get them submitted. Eggzackle ... my point is that I see the win32 train leaving the station pretty soon, and I don't see anyone else ready to get on board. Not quite yet - yesterday I got hold of a windows box for the first time in months and had significant building problems (the symlink problem and others). This needs to be robust. I will be sending in some fixes in due course. Is it time to set a tentative target date for a feature freeze, though? Perhaps some time in mid to late June would be good. That way people could work to a known timeline. cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] 7.5 features
Andrew Dunstan wrote: Tom Lane said: Bruce Momjian [EMAIL PROTECTED] writes: What we really need is for these folks to start finalizing their patches and get them submitted. Eggzackle ... my point is that I see the win32 train leaving the station pretty soon, and I don't see anyone else ready to get on board. Not quite yet - yesterday I got hold of a windows box for the first time in months and had significant building problems (the symlink problem and others). This needs to be robust. I will be sending in some fixes in due course. Is it time to set a tentative target date for a feature freeze, though? Perhaps some time in mid to late June would be good. That way people could work to a known timeline. Yes, that was the timeline I was considering too. Folks, we can be sure you have until the end of May to complete items, but after that, we don't know how soon the feature freeze will happen. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PITR Phase 2 - Design Planning
Simon Riggs wrote: I'd suggest extending the CHECKPOINT command so you can say: CHECKPOINT text message e.g. CHECKPOINT 'starting payroll Feb04'; (I'm sure some other DBMS does this...head spinning can;t recall...) the text could just appear in the xlog record data packet... I believe you are thinking of a savepoint rather than a checkpoint. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] 7.5 features
On Thu, 29 Apr 2004, Andrew Dunstan wrote: Tom Lane said: Bruce Momjian [EMAIL PROTECTED] writes: What we really need is for these folks to start finalizing their patches and get them submitted. Eggzackle ... my point is that I see the win32 train leaving the station pretty soon, and I don't see anyone else ready to get on board. Not quite yet - yesterday I got hold of a windows box for the first time in months and had significant building problems (the symlink problem and others). This needs to be robust. I will be sending in some fixes in due course. Is it time to set a tentative target date for a feature freeze, though? Perhaps some time in mid to late June would be good. That way people could work to a known timeline. Right now, the feature freeze is tentative for 1st of June, which has been thrown around a few times already ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] 7.5 features
Marc G. Fournier wrote: Right now, the feature freeze is tentative for 1st of June, which has been thrown around a few times already ... If it has I've missed it - always seemed somewhat vaguer to me. Thanks andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] 7.5 features
Marc G. Fournier wrote: On Thu, 29 Apr 2004, Andrew Dunstan wrote: Tom Lane said: Bruce Momjian [EMAIL PROTECTED] writes: What we really need is for these folks to start finalizing their patches and get them submitted. Eggzackle ... my point is that I see the win32 train leaving the station pretty soon, and I don't see anyone else ready to get on board. Not quite yet - yesterday I got hold of a windows box for the first time in months and had significant building problems (the symlink problem and others). This needs to be robust. I will be sending in some fixes in due course. Is it time to set a tentative target date for a feature freeze, though? Perhaps some time in mid to late June would be good. That way people could work to a known timeline. Right now, the feature freeze is tentative for 1st of June, which has been thrown around a few times already ... Yea, I think I remember that date being mentioned. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] Call for 7.5 feature completion
Andrew Dunstan wrote: Marc G. Fournier wrote: Right now, the feature freeze is tentative for 1st of June, which has been thrown around a few times already ... If it has I've missed it - always seemed somewhat vaguer to me. Yes, it was vague. The question is now that we are a month away, do we want to target June 1, mid-June, or July 1. Some are saying that once Win32 is ready, it will justify a release even if the other features are not ready. Looking at the list: o tablespaces (Gavin) o nested transactions (Alvaro) o two-phase commit (Heikki Linnakangas) o integrated pg_autovacuum (O'Connor) o PITR (Riggs) o Win32 (Claudio, Magnus) o replication (Sloney) o server-side Java I have talked to each person, and here is where we are on these projects: o tablespaces (Gavin) Gavin has the code 3/4 done. I told him to submit soon and he said we will have something in a few days. o nested transactions (Alvaro) Has submitted patches that are under review. He has the nesting of BEGIN done, and storage manager subtransaction handling. I think he needs to do pg_subtrans system table and error recovery. He says he can finish in time. o two-phase commit (Heikki Linnakangas) Has patches and is waiting for Alvaro's patches to be applied. Not sure how complete his patch is but it has been posted. We need to review its functionality. o integrated pg_autovacuum (O'Connor) Unknown. We need fixes for temp tables for 7.4.3 too. o PITR (Riggs) Patch submitted but now discussing how logging should be accomplished. o Win32 (Claudio, Magnus) Win32 status page lists some items still to be completed: http://momjian.postgresql.org/main/writings/pgsql/project o replication (Sloney) Released independently. Nearing beta. o server-side Java Released independently. Nearing beta or already in beta. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Call for 7.5 feature completion
Bruce Momjian wrote: Yes, it was vague. The question is now that we are a month away, do we want to target June 1, mid-June, or July 1. Some are saying that once Win32 is ready, it will justify a release even if the other features are not ready. I think we should have this conversation once the win32 code is ready. Since the beta period / release schedule usually takes several months in and of it self, anyone working on an item from the list below is going to lose a lot of steam if they are 80% done but get excluded from 7.5 and have to wait out the release process. Looking at the list: o tablespaces (Gavin) o nested transactions (Alvaro) o two-phase commit (Heikki Linnakangas) o integrated pg_autovacuum (O'Connor) o PITR (Riggs) o Win32 (Claudio, Magnus) o replication (Sloney) o server-side Java o integrated pg_autovacuum (O'Connor) Unknown. We need fixes for temp tables for 7.4.3 too. Yes I know, starting next week, I will have some time to work on this. I will fix the temp table problem first, then the 7.5 only work. FYI, I do fully intend on getting pg_autovacuum integrated for 7.5, even with a June 1 deadline. Matthew O'Connor ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PITR Phase 2 - Design Planning
On Thu, Apr 29, 2004 at 05:09:19PM +0200, Peter Eisentraut wrote: Simon Riggs wrote: I'd suggest extending the CHECKPOINT command so you can say: CHECKPOINT text message e.g. CHECKPOINT 'starting payroll Feb04'; (I'm sure some other DBMS does this...head spinning can;t recall...) the text could just appear in the xlog record data packet... I believe you are thinking of a savepoint rather than a checkpoint. But a savepoint has a very precise meaning in the SQL standard, which relates to points in a transaction you can roll back to. I don't think you want to overload with this other meaning, which I see as putting a special mark in the XLog -- completely unrelated. ISTM Checkpoint 'something' would flush all xlogs, insert some kind of special xlog record with 'something' on it, and then archive everything (up to that point?). -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) XML! Exclaimed C++. What are you doing here? You're not a programming language. Tell that to the people who use me, said XML. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 29 Apr 2004, Bruce Momjian wrote: Andrew Dunstan wrote: Marc G. Fournier wrote: Right now, the feature freeze is tentative for 1st of June, which has been thrown around a few times already ... If it has I've missed it - always seemed somewhat vaguer to me. Yes, it was vague. The question is now that we are a month away, do we want to target June 1, mid-June, or July 1. Target June 1st ... when May 29th or so rolls around, we can then determine whether or not we need to extend it, and why. We've gone down this road too many times, where we've extended cause a patch was right around the corner that finally came in a month later ... The point of a deadline is that you don't make it floating ... if the deadline arrives, and someone can justify extending it a few days to a week, then so be it ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] PITR Phase 2 - Design Planning
On Thu, 2004-04-29 at 16:09, Peter Eisentraut wrote: Simon Riggs wrote: I'd suggest extending the CHECKPOINT command so you can say: CHECKPOINT text message e.g. CHECKPOINT 'starting payroll Feb04'; (I'm sure some other DBMS does this...head spinning can;t recall...) the text could just appear in the xlog record data packet... I believe you are thinking of a savepoint rather than a checkpoint. Perhaps that was the inspiration, but no, I definitely meant a CHECKPOINT. But now you come to mention it, it would be better just to have a command that simply wrote a named record to the xlog, so it can be searched for later... COMMENT [IN TRANSACTION LOG] 'starting payroll Feb04' Best Regards, Simon Riggs ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] PITR logging control program
On Thu, 2004-04-29 at 15:22, Bruce Momjian wrote: Alvaro Herrera wrote: On Thu, Apr 29, 2004 at 10:07:01AM -0400, Bruce Momjian wrote: Alvaro Herrera wrote: Is the API able to indicate a written but not-yet-filled WAL segment? So an archiver could copy the filled part, and refill it later. This may be needed because a segment could take a while to be filled. I couldn't figure that out, but I don't think it does. It would have to lock the WAL writes so it could get a good copy, I think, and I didn't see that. I'm not sure but I don't think so. You don't have to lock the WAL for writing, because it will always write later in the file than you are allowed to read. (If you read more than you were told to, it's your fault as an archiver.) My point was that without locking the WAL, we might get part of a WAL write in our file, but I now realize that during a crash the same thing might happen, so it would be OK to just copy it even if it is being written to. Simon posted the rest of his patch that shows changes to the backend, and a comment reads: + * The name of the notification file is the message that will be picked up + * by the archiver, e.g. we write RLogDir/000100C6.full + * and the archiver then knows to archive XLOgDir/000100C6, + * while it is doing so it will rename RLogDir/000100C6.full + * to RLogDir/000100C6.busy, then when complete, rename it again + * to RLogDir/000100C6.done so it is only archiving full logs. Also, I think this archiver should be able to log to a local drive, network drive (trivial), tape drive, ftp, or use an external script to transfer the logs somewhere. (ftp would probably be an external script with 'expect'). Bruce is correct, the API waits for the archive to be full before archiving. I had thought about the case for partial archiving: basically, if you want to archive in smaller chunks, make your log files smaller...this is now a compile time option. Possibly there is an argument to make the xlog file size configurable, as a way of doing what you suggest. Taking multiple copies of the same file, yet trying to work out which one to apply sounds complex and error prone to me. It also increases the cost of the archival process and thus drains other resources. The archiver should be able to do a whole range of things. Basically, that point was discussed and the agreed approach was to provide an API that would allow anybody and everybody to write whatever they wanted. The design included pg_arch since it was clear that there would be a requirement in the basic product to have those facilities - and in any case any practically focused API has a reference port as a way of showing how to use it and exposing any bugs in the server side implementation. The point is...everybody is now empowered to write tape drive code, whatever you fancy go do. Best regards, Simon Riggs ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Call for 7.5 feature completion
Marc G. Fournier wrote: On Thu, 29 Apr 2004, Bruce Momjian wrote: Andrew Dunstan wrote: Marc G. Fournier wrote: Right now, the feature freeze is tentative for 1st of June, which has been thrown around a few times already ... If it has I've missed it - always seemed somewhat vaguer to me. Yes, it was vague. The question is now that we are a month away, do we want to target June 1, mid-June, or July 1. Target June 1st ... when May 29th or so rolls around, we can then determine whether or not we need to extend it, and why. We've gone down this road too many times, where we've extended cause a patch was right around the corner that finally came in a month later ... The point of a deadline is that you don't make it floating ... if the deadline arrives, and someone can justify extending it a few days to a week, then so be it ... No. We tried that in the past and we ended up extending it in pieces several times. The effect was that we delayed feature freeze by a month or two, and other features never got developed in that timeframe. I remember SMP fixes for 7.3 as causing that problem. We didn't do that for 7.4 and it was better. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Call for 7.5 feature completion
Merlin Moncure wrote: Bruce Momjian wrote: Yes, it was vague. The question is now that we are a month away, do we want to target June 1, mid-June, or July 1. If I may humbly chime in here...there currently is no binary packing for the win32 port. Magnus is currently working on an installer/service manager (dubbed 'longer term' on the status page) that would be nice to see make it in before the feature freeze. Based on the current rate of development, finishing this and various other sundry issues (even allowing for corrections within the beta period) within a month involves a fair degree of optimism. In short, speaking strictly from a win32 perspective, a June 1 date will probably be missed. Well, if Win32 doesn't complete by June 1, do we still do the feature freeze? I don't want to be adding features after the freeze, that is for sure. When we have done that in the past, it has caused problems because some stuff gets in to make the system unstable, but other stuff gets left out. As I remember, we decided that we should not make decisions to extend the feature freeze date just before the freeze date because it causes people to stop doing development, then we extend. I think we should go another week or two then decide if we should stay to June 1, and if we don't, we should schedule for July 1. Extending it by a week at a time, and almost before the deadline, has caused considerable waste of time in the past. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Call for 7.5 feature completion
On Thu, Apr 29, 2004 at 01:26:07PM -0400, Bruce Momjian wrote: o nested transactions (Alvaro) Has submitted patches that are under review. He has the nesting of BEGIN done, and storage manager subtransaction handling. I think he needs to do pg_subtrans system table and error recovery. He says he can finish in time. o two-phase commit (Heikki Linnakangas) Has patches and is waiting for Alvaro's patches to be applied. Not sure If 2PC patches are ready and nested transactions not, is it worth considering applying 2PC when subtransactions aren't ready? I appreciate the work that will cause, so I'm not saying this lightly. It's probably too early to make that decision now, but I'd suggest keeping it in mind in case something goes off the rails. A -- Andrew Sullivan | [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] PITR Phase 2 - Design Planning
Simon Riggs wrote: On Thu, 2004-04-29 at 16:09, Peter Eisentraut wrote: Perhaps that was the inspiration, but no, I definitely meant a CHECKPOINT. But now you come to mention it, it would be better just to have a command that simply wrote a named record to the xlog, so it can be searched for later... COMMENT [IN TRANSACTION LOG] 'starting payroll Feb04' FWFW, Oracle's COMMIT syntax has an optional COMMENT specifier which is used for documenting a distributed transaction. In-doubt transactions can then be manually committed or aborted by referencing the transaction associated with the comment. Example: COMMIT WORK COMMENT 'A complex distributed Tx'; Perhaps there is some common ground between the 2PC implementation and PITR? Mike Mascari ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] PITR logging control program
Simon Riggs wrote: Also, I think this archiver should be able to log to a local drive, network drive (trivial), tape drive, ftp, or use an external script to transfer the logs somewhere. (ftp would probably be an external script with 'expect'). Bruce is correct, the API waits for the archive to be full before archiving. I had thought about the case for partial archiving: basically, if you want to archive in smaller chunks, make your log files smaller...this is now a compile time option. Possibly there is an argument to make the xlog file size configurable, as a way of doing what you suggest. Taking multiple copies of the same file, yet trying to work out which one to apply sounds complex and error prone to me. It also increases the cost of the archival process and thus drains other resources. The archiver should be able to do a whole range of things. Basically, that point was discussed and the agreed approach was to provide an API that would allow anybody and everybody to write whatever they wanted. The design included pg_arch since it was clear that there would be a requirement in the basic product to have those facilities - and in any case any practically focused API has a reference port as a way of showing how to use it and exposing any bugs in the server side implementation. The point is...everybody is now empowered to write tape drive code, whatever you fancy go do. Agreed we want to allow the superuser control over writing of the archive logs. The question is how do they get access to that. Is it by running a client program continuously or calling an interface script from the backend? My point was that having the backend call the program has improved reliablity and control over when to write, and easier administration. How are people going to run pg_arch? Via nohup? In virtual screens? If I am at the console and I want to start it, do I use ? If I want to stop it, do I do a 'ps' and issue a 'kill'? This doesn't seem like a good user interface to me. To me the problem isn't pg_arch itself but the idea that a client program is going to be independently finding(polling) and copying of the archive logs. I am thinking the client program is called with two arguments, the xlog file name, and the arch location defined in GUC. Then the client program does the write. The problem there though is who gets the write error since the backend will not wait around for completion? Another case is server start/stop. You want to start/stop the archive logger to match the database server, particularly if you reboot the server. I know Informix used a client program for logging, and it was a pain to administer. I would be happy with an exteral program if it was started/stoped by the postmaster (or via GUC change) and received a signal when a WAL file was written. But if we do that, it isn't really an external program anymore but another child process like our stats collector. I am willing to work on this if folks think this is a better approach. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Call for 7.5 feature completion
Bruce Momjian wrote: Well, if Win32 doesn't complete by June 1, do we still do the feature freeze? I don't want to be adding features after the freeze, that is for sure. When we have done that in the past, it has caused problems because some stuff gets in to make the system unstable, but other stuff gets left out. It seams as though the win32 port is driving the release schedule, so instead of picking June 1 out of the air, how about getting an estimated time for completion from them, or more specifically a date by which they should have all the major features done? I know it's a chicken and egg problem, do we set a date for developers to shoot for, or do shoot for specific features and choose a date from there. I think there can no hard and fast rule on this, it depends on the feature and the desire to put it in place. Matthew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Call for 7.5 feature completion
Andrew Sullivan wrote: On Thu, Apr 29, 2004 at 01:26:07PM -0400, Bruce Momjian wrote: o nested transactions (Alvaro) Has submitted patches that are under review. He has the nesting of BEGIN done, and storage manager subtransaction handling. I think he needs to do pg_subtrans system table and error recovery. He says he can finish in time. o two-phase commit (Heikki Linnakangas) Has patches and is waiting for Alvaro's patches to be applied. Not sure If 2PC patches are ready and nested transactions not, is it worth considering applying 2PC when subtransactions aren't ready? I appreciate the work that will cause, so I'm not saying this lightly. It's probably too early to make that decision now, but I'd suggest keeping it in mind in case something goes off the rails. 2-PC hasn't actually been posted for applicaiton, while Alvaro's have. They know they will conflict and 2PC wanted to wait for Alvaro's changes. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Call for 7.5 feature completion
Tom Lane wrote: Merlin Moncure [EMAIL PROTECTED] writes: Bruce Momjian wrote: Yes, it was vague. The question is now that we are a month away, do we want to target June 1, mid-June, or July 1. ... In short, speaking strictly from a win32 perspective, a June 1 date will probably be missed. Fair enough. Do you feel comfortable with either of the other dates? The point of this discussion is not so much to crack the whip as to get people's schedule expectations aligned. If we set a date X, those who can't possibly meet it will know to plan on getting into a later release instead, and those who can meet it if they hustle will know to start hustling. But we need to set a date. It's unfair to other parts of the project to say oh, we'll wait till Win32 is ready and then give two weeks notice. Well, looking at Win32, we are making steady progress, but it is uncertain if we will make June 1, and we might not know even on May 15 if we will make that date, and if we don't know we have to either stick to June 1 and ignore Win32 or extend to July 1. We can't decide on May 29 because then all the other features are left hanging to see what happens with Win32. I wonder if we should just pick July 1 because there is good expectation based on current progress that Win32 will be done by June 15, which would be the next cuttoff date. Of course we can wait until May 15 and then decide. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Call for 7.5 feature completion
Merlin Moncure [EMAIL PROTECTED] writes: Bruce Momjian wrote: Yes, it was vague. The question is now that we are a month away, do we want to target June 1, mid-June, or July 1. ... In short, speaking strictly from a win32 perspective, a June 1 date will probably be missed. Fair enough. Do you feel comfortable with either of the other dates? The point of this discussion is not so much to crack the whip as to get people's schedule expectations aligned. If we set a date X, those who can't possibly meet it will know to plan on getting into a later release instead, and those who can meet it if they hustle will know to start hustling. But we need to set a date. It's unfair to other parts of the project to say oh, we'll wait till Win32 is ready and then give two weeks notice. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Call for 7.5 feature completion
Matthew T. O'Connor wrote: Bruce Momjian wrote: Well, if Win32 doesn't complete by June 1, do we still do the feature freeze? I don't want to be adding features after the freeze, that is for sure. When we have done that in the past, it has caused problems because some stuff gets in to make the system unstable, but other stuff gets left out. It seams as though the win32 port is driving the release schedule, so instead of picking June 1 out of the air, how about getting an estimated time for completion from them, or more specifically a date by which they should have all the major features done? I just posted on this. I am putting timezone in today, but we have fsync to do, and some other stuff. Might be done in two weeks, but most likely it will take 4. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Call for 7.5 feature completion
Bruce Momjian wrote: Well, if Win32 doesn't complete by June 1, do we still do the feature freeze? I don't want to be adding features after the freeze, that is [...] As I remember, we decided that we should not make decisions to extend the feature freeze date just before the freeze date because it causes people to stop doing development, then we extend. I think we should go another week or two then decide if we should stay to June 1, and if we don't, we should schedule for July 1. Extending it by a week at a time, and almost before the deadline, has caused considerable waste of time in the past. I agree 100%. Right now the win32 side doesn't qualify for beta...just the date issue alone is a pretty big deal, IMO. There are pending patches for only about 50% of the outstanding issues. A tremendous amount of work has been done, but there is still quite a bit to be done to meet basic QC guidelines. So I suggest (my choices are of course subjective): Dividing win32 'should fix' (installer, /contrib, etc.) win32 'must fix' (psql query cancel, 1970 dates, non-cygwin regression) Apply freeze date to the 'must fix' items. 'Should fix' items can be delayed until the beta, dot release, or beyond. My personal estimation on completion date depends on what gets put in which category. If everything is designated 'must fix', I think July 1 is practical. Merlin ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 29 Apr 2004, Bruce Momjian wrote: Merlin Moncure wrote: Bruce Momjian wrote: Yes, it was vague. The question is now that we are a month away, do we want to target June 1, mid-June, or July 1. If I may humbly chime in here...there currently is no binary packing for the win32 port. Magnus is currently working on an installer/service manager (dubbed 'longer term' on the status page) that would be nice to see make it in before the feature freeze. Based on the current rate of development, finishing this and various other sundry issues (even allowing for corrections within the beta period) within a month involves a fair degree of optimism. In short, speaking strictly from a win32 perspective, a June 1 date will probably be missed. Well, if Win32 doesn't complete by June 1, do we still do the feature freeze? I don't want to be adding features after the freeze, that is for sure. When we have done that in the past, it has caused problems because some stuff gets in to make the system unstable, but other stuff gets left out. That is something that should be discussed a couple of days before June 1st, when we have a clearer idea of just how far off Win32 will be ... right now, if Win32 is *that* far off the June 1st mark, I'd say definitely to a feature freeze, as much as I'd like to see the Win32 features in ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 29 Apr 2004, Bruce Momjian wrote: No. We tried that in the past and we ended up extending it in pieces several times. The effect was that we delayed feature freeze by a month or two, and other features never got developed in that timeframe. I remember SMP fixes for 7.3 as causing that problem. We didn't do that for 7.4 and it was better. *If* June 1st comes along, and Win32 isn't ready, there is nothing wrong with freezing the code *except* for a pending Win32 patch ... but, such a focused extension would depend on how close to being ready that pending patch is ... Again, something that cannot be accurately determined until June 1st, and by even speculating on extending, we are just makign it easier to shrug and say not a big rush, we'll most likely extend the freeze anyway ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 29 Apr 2004, Matthew T. O'Connor wrote: I know it's a chicken and egg problem, do we set a date for developers to shoot for, or do shoot for specific features and choose a date from there. I think there can no hard and fast rule on this, it depends on the feature and the desire to put it in place. If we set the date based on a specific featuer, v7.4 would still be in development ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] PITR Phase 2 - Design Planning
Alvaro Herrera wrote: But a savepoint has a very precise meaning in the SQL standard, which relates to points in a transaction you can roll back to. I don't think you want to overload with this other meaning, which I see as putting a special mark in the XLog -- completely unrelated. They are completely unrelated because you're considering them on two different levels. From the user interface level they are both a place to roll back to. Whether they are inside the same transaction or not is like the difference between a normal and a holdable cursor, but it's the same interface. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 29 Apr 2004, Bruce Momjian wrote: I wonder if we should just pick July 1 because there is good expectation based on current progress that Win32 will be done by June 15, which would be the next cuttoff date. Of course we can wait until May 15 and then decide. Start of summer holidays, when most ppl will want to be away? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] PITR logging control program
On Thu, Apr 29, 2004 at 07:34:47PM +0100, Simon Riggs wrote: Bruce is correct, the API waits for the archive to be full before archiving. I had thought about the case for partial archiving: basically, if you want to archive in smaller chunks, make your log files smaller...this is now a compile time option. Possibly there is an argument to make the xlog file size configurable, as a way of doing what you suggest. Taking multiple copies of the same file, yet trying to work out which one to apply sounds complex and error prone to me. It also increases the cost of the archival process and thus drains other resources. My idea was basically that the archiver could be told I've finished writing XLog segment 1 until byte 9000, so the archiver would dd if=xlog-1 seek=0 skip=0 bs=1c count=9000c of=archive-1 And later, it would get a notification segment 1 until byte 18000 he does dd if=xlog-1 seek=0 skip=0 bs=1c count=18000c of=archive-1 Or, if it's smart enough, dd if=xlog-1 seek=9000c skip=9000c bs=1c count=9000c of=archive-1 Basically it is updating the logs as soon as it receives the notifications. Writing 16 MB of xlogs could take some time. When a full xlog segment has been written, a different kind of notification can be issued. A dumb archiver could just ignore the incremental ones and copy the files only upon receiving this other kind. I think that if log files are too small, maybe it will be a waste of resources (which ones?). Anyway, it's just an idea. -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 29 Apr 2004, Merlin Moncure wrote: Bruce Momjian wrote: Well, if Win32 doesn't complete by June 1, do we still do the feature freeze? I don't want to be adding features after the freeze, that is [...] As I remember, we decided that we should not make decisions to extend the feature freeze date just before the freeze date because it causes people to stop doing development, then we extend. I think we should go another week or two then decide if we should stay to June 1, and if we don't, we should schedule for July 1. Extending it by a week at a time, and almost before the deadline, has caused considerable waste of time in the past. I agree 100%. Right now the win32 side doesn't qualify for beta...just the date issue alone is a pretty big deal, IMO. There are pending patches for only about 50% of the outstanding issues. A tremendous amount of work has been done, but there is still quite a bit to be done to meet basic QC guidelines. So I suggest (my choices are of course subjective): Dividing win32 'should fix' (installer, /contrib, etc.) win32 'must fix' (psql query cancel, 1970 dates, non-cygwin regression) actually, IMHO, some of these should be classified as bug fixes, assuming that the patches are already in the source tree, and don't affect the feature freeze in any way, only the release ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Call for 7.5 feature completion
-Original Message- From: Merlin Moncure [mailto:[EMAIL PROTECTED] Sent: Thursday, April 29, 2004 12:45 PM To: Bruce Momjian Cc: [EMAIL PROTECTED] Subject: Re: [HACKERS] Call for 7.5 feature completion Bruce Momjian wrote: Well, if Win32 doesn't complete by June 1, do we still do the feature freeze? I don't want to be adding features after the freeze, that is [...] As I remember, we decided that we should not make decisions to extend the feature freeze date just before the freeze date because it causes people to stop doing development, then we extend. I think we should go another week or two then decide if we should stay to June 1, and if we don't, we should schedule for July 1. Extending it by a week at a time, and almost before the deadline, has caused considerable waste of time in the past. I agree 100%. Right now the win32 side doesn't qualify for beta...just the date issue alone is a pretty big deal, IMO. There are pending patches for only about 50% of the outstanding issues. A tremendous amount of work has been done, but there is still quite a bit to be done to meet basic QC guidelines. So I suggest (my choices are of course subjective): Dividing win32 'should fix' (installer, /contrib, etc.) win32 'must fix' (psql query cancel, 1970 dates, non-cygwin regression) About the date/time stuff. We (of course) had the exact same issues with dates and times when we did a native port of 7.1.3. Since we want to represent dates and times with exactness of nanoseconds from thousands of years BC up to thousands of years in the future, it was a major piece of work. The code base is not titanically large, but it was pretty tricky getting it right. First the bad news: There is no native data type in the C lanaguage that can store with the required precision so we used Moshier's Qfloat. With PostgreSQL, perhaps the intermediate values could be stored in numeric type instead. Now the worse news: Everything is in C++ and represented as objects Maybe it would not be too hard to translate the algorithms into C, since the languages are fairly similar. Apply freeze date to the 'must fix' items. 'Should fix' items can be delayed until the beta, dot release, or beyond. My personal estimation on completion date depends on what gets put in which category. If everything is designated 'must fix', I think July 1 is practical. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PITR logging control program
On Thu, 2004-04-29 at 20:24, Bruce Momjian wrote: Simon Riggs wrote: The archiver should be able to do a whole range of things. Basically, that point was discussed and the agreed approach was to provide an API that would allow anybody and everybody to write whatever they wanted. The design included pg_arch since it was clear that there would be a requirement in the basic product to have those facilities - and in any case any practically focused API has a reference port as a way of showing how to use it and exposing any bugs in the server side implementation. The point is...everybody is now empowered to write tape drive code, whatever you fancy go do. Agreed we want to allow the superuser control over writing of the archive logs. The question is how do they get access to that. Is it by running a client program continuously or calling an interface script from the backend? My point was that having the backend call the program has improved reliablity and control over when to write, and easier administration. Agreed. We've both suggested ways that can occur, though I suggest this is much less of a priority, for now. Not no, just not now. How are people going to run pg_arch? Via nohup? In virtual screens? If I am at the console and I want to start it, do I use ? If I want to stop it, do I do a 'ps' and issue a 'kill'? This doesn't seem like a good user interface to me. To me the problem isn't pg_arch itself but the idea that a client program is going to be independently finding(polling) and copying of the archive logs. I am thinking the client program is called with two arguments, the xlog file name, and the arch location defined in GUC. Then the client program does the write. The problem there though is who gets the write error since the backend will not wait around for completion? Another case is server start/stop. You want to start/stop the archive logger to match the database server, particularly if you reboot the server. I know Informix used a client program for logging, and it was a pain to administer. pg_arch is just icing on top of the API. The API is the real deal here. I'm not bothered if pg_arch is not accepted, as long as we can adopt the API. As noted previously, my original mind was to split the API away from the pg_arch application to make it clearer what was what. Once that has been done, I encourage others to improve pg_arch - but also to use the API to interface with other BAR prodiucts. If you're using PostgreSQL for serious business then you will be using a serious BAR product as well. There are many FOSS alternatives... The API's purpose is to allow larger, pre-existing BAR products to know when and how to retrieve data from PostgreSQL. Those products don't and won't run underneath postmaster, so although I agree with Peter's original train of thought, I also agree with Tom's suggestion that we need an API more than we need an archiver process. I would be happy with an exteral program if it was started/stoped by the postmaster (or via GUC change) and received a signal when a WAL file was written. That is exactly what has been written. The PostgreSQL side of the API is written directly into the backend, in xlog.c and is therefore activated by postmaster controlled code. That then sends a signal to the process that will do the archiving - the Archiver side of the XLogArchive API has it as an in-process library. (The signal is, in fact, a zero-length file written to disk because there are many reasons why an external archiver may not be ready to archive or even up and running to receive a signal). The only difference is that there is some confusion as to the role and importance of pg_arch. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] PITR logging control program
On Thu, 2004-04-29 at 20:24, Bruce Momjian wrote: I am willing to work on this... There is much work still to be done to make PITR work..accepting all of the many comments made. If anybody wants this by 1 June, I think we'd better look sharp. My aim has been to knock one of the URGENT items on the TODO list into touch, however that was to be achieved. The following work remains...from all that has been said... - halt restore at particular condition (point in time, txnid etc) - archive policy to control whether to halt database should archiving fail and space run out (as Oracle, Db2 do), or not (as discussed) - cope with restoring a stream of logs larger than the disk space on the restoration target system - integrate restore with tablespace code, to allow tablespace backups - build XLogSpy mechanism to allow DBA to better know when to recover to - extend logging mechanism to allow recovery time prediction - publicise the API with BAR open source teams, to get feedback and to encourage them to use the API to allow PostgreSQL support for their BAR - use the API to build interfaces to the 100+ BAR products on the market - performance tuning of xlogs, to ensure minimum xlog volume written - performance tuning of recovery, to ensure wasted effort avoided - allow archiver utility to be managed by postmaster - write some good documentation - comprehensive crash testing - really comprehensive crash testing - very comprehensive crash testing It seems worth working on things in some kind of priority order. I claim these, by the way, but many others look important and interesting to me: - halt restore at particular condition (point in time, txnid etc) - cope with restoring a stream of logs larger than the disk space on the restoration target system - write some good documentation Best Regards, Simon Riggs ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 29 Apr 2004, Simon Riggs wrote: What I would add is: if PITR and Win32 do make it into the release, I would strongly urge for an extended beta period. It would not prove good press if 100,000 new Windows users tripped over on various issues, nor even 1 press-worthy user was unable to recover correctly. Granted ... our beta periods always tend to be as long as required to be confident of stability ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 2004-04-29 at 20:50, Marc G. Fournier wrote: On Thu, 29 Apr 2004, Bruce Momjian wrote: Merlin Moncure wrote: Bruce Momjian wrote: Yes, it was vague. The question is now that we are a month away, do we want to target June 1, mid-June, or July 1. If I may humbly chime in here...there currently is no binary packing for the win32 port. Magnus is currently working on an installer/service manager (dubbed 'longer term' on the status page) that would be nice to see make it in before the feature freeze. Based on the current rate of development, finishing this and various other sundry issues (even allowing for corrections within the beta period) within a month involves a fair degree of optimism. In short, speaking strictly from a win32 perspective, a June 1 date will probably be missed. Well, if Win32 doesn't complete by June 1, do we still do the feature freeze? I don't want to be adding features after the freeze, that is for sure. When we have done that in the past, it has caused problems because some stuff gets in to make the system unstable, but other stuff gets left out. That is something that should be discussed a couple of days before June 1st, when we have a clearer idea of just how far off Win32 will be ... right now, if Win32 is *that* far off the June 1st mark, I'd say definitely to a feature freeze, as much as I'd like to see the Win32 features in ... Completely agree - I've been working to that timescale since March. This is probably the determining factor as to whether the release number is 7.5 or 8.0? What I would add is: if PITR and Win32 do make it into the release, I would strongly urge for an extended beta period. It would not prove good press if 100,000 new Windows users tripped over on various issues, nor even 1 press-worthy user was unable to recover correctly. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Call for 7.5 feature completion
Marc G. Fournier [EMAIL PROTECTED] writes: On Thu, 29 Apr 2004, Merlin Moncure wrote: So I suggest (my choices are of course subjective): Dividing win32 'should fix' (installer, /contrib, etc.) win32 'must fix' (psql query cancel, 1970 dates, non-cygwin regression) actually, IMHO, some of these should be classified as bug fixes, assuming that the patches are already in the source tree, and don't affect the feature freeze in any way, only the release ... The issue is really whether the bug fixes are major enough to invalidate any previous beta testing. For example, it would be unreasonable to go beta with the 1970 dates issue unresolved, because what we are talking about doing to fix that involves plopping in a pretty big chunk of new (to us anyway) code. And since we plan to use that code on all platforms, it needs beta testing everywhere. Another example is the sync/fsync changes we are talking about making --- that's pretty basic and it'll need QA on all platforms not just win32. It would be reasonable to go beta with open minor issues that we expect to be fixable by localized patches. (But on the other hand, why wouldn't such issues be resolved before then?) So I'm not sure that should fix vs must fix is the appropriate classification. Small/localized fix vs nontrivial change is more what I'm worried about. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] Small OS ports Handheld devices
Seemingly completely opposite to my usual focus on large scale enterprise issues and requirements, I'm looking into PostgreSQL on small handheld devices. Clearly these have limited memory and little disk capability... Are there some ports available to various devices? What is the lowest memory footprint PostgreSQL has achieved? [8Mb was last quote] How little disk space has anyone achieved? Is that an available port, or just a set of configure options? Is there a definitive list of supported ports? (for anything) Does PostgreSQL run on? - Windows 2003 (is that part of Win32 port as a defined target?) - PalmOS - Zaurus Related info is copied below: Q: Does PostgreSQL write repeatedly even when there is a no overt SQL write activity? With a write-capable, but very low write rate, PostgreSQL seems a good candidate, given current memory capacities? Dredging up last year's discussion on postgres on a PDA we have.. Al Bean albean84 ( at ) hotmail ( dot ) com writes: My PDA has 32MB of RAM (well more like 16 with the way Sharp allocates it) and I use a 256MB FLASH memory card which sort of looks like a hard drive on the PDA. I know postgresql will run in about 8MB of RAM so I think I should be ok on this front but I'm wondering about the data residing in FLASH. Postgres has kind of gotten away from the notion of a small disk footprint :-(. Some things to look at: * Definitely reduce the size of WAL segments (see XLogSegSize). You can't afford the default 16MB. * You probably don't want to have multiple databases. I'd suggest doing all your work in template1. Consider also dropping template0 --- its only value would be to reinit template1, and if you need to do that you might as well re-initdb. But I fear the real problem is going to be that FLASH memory has a limited lifetime (measured in write cycles). Postgres is going to spend that lifetime with abandon, because it has absolutely no motivation to avoid disk writes. Are you prepared to replace the FLASH every so often? ..reply... In article 1883 ( dot ) 1042606899 ( at ) sss ( dot ) pgh ( dot ) pa ( dot ) us, Tom Lane tgl ( at ) sss ( dot ) pgh ( dot ) pa ( dot ) us wrote: attention I think it was on the order of magnitude of 1 write cycles --- which Postgres could blow through in no time. I hope it's better now, but I dunno by how much. Anyone have more up-to-date info? Only about 1 order of magnitude better. From the Linux Embedded FAQ: Q: 3.3 Flash Limited write cycles Flash have limited write cycles capabilities from 200 000 to 400 000. Using swap on such device is dangerous. 300 000 writes gives you 200 days at 1 write / minute and 83 hours at 1 write / second. More, If you interrupt power at arbitrary times while the device is writing, you can lose the integrity of the file system being modified. The loss is not limited to the 512 byte sector being modified, as it generally is with rotating disks; you can lose an entire erase block, maybe 64K at once. The risk goes up as the disk approaches full, because erase rewrite cycles happen more often in this condition. Un*x file systems spread their super block tables around the disk, in the hope that at least one will survive a head crash. Create one file, then /bin/sync: what percentage of the device's erase blocks get hit? Thanks, Simon Riggs 2nd Quadrant ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Call for 7.5 feature completion
Marc G. Fournier [EMAIL PROTECTED] writes: *If* June 1st comes along, and Win32 isn't ready, there is nothing wrong with freezing the code *except* for a pending Win32 patch ... Yeah there is ... In the first place it's unfair to other developers to make schedule slips at the last moment, and especially to *plan* to do so. I would prefer we determine at least two weeks, preferably a month, in advance whether there is going to be a schedule change. I think it's entirely appropriate to decide now whether we are going to hold to June 1 feature freeze or slip it. That gives other people a chance to do useful work while we approach the deadline. In the second, any Win32 issues that remain unresolved as of June 1 are likely to be major enough (in terms of code impact) that we can't realistically contemplate slipping in post-beta patches for them. The fallback position is going to have to be no (supported) Win32 port in 7.5, not we'll fix this in beta. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Small OS ports Handheld devices
Simon Riggs [EMAIL PROTECTED] writes: ... I'm looking into PostgreSQL on small handheld devices. Clearly these have limited memory and little disk capability... Are there some ports available to various devices? I don't know of any supported ports. What is the lowest memory footprint PostgreSQL has achieved? This depends entirely on your postgresql.conf settings and how complex the queries you want to process are. I would think you could get it down to maybe 4 or so meg if you have bottom-of-the-barrel requirements. Performance in this configuration not guaranteed ;-) How little disk space has anyone achieved? Is that an available port, or just a set of configure options? IIRC, configuring the WAL segment size is a pg_config_manual.h setting in CVS tip, but in extant releases you'd have to dig into the xlog code to adjust it. Q: Does PostgreSQL write repeatedly even when there is a no overt SQL write activity? Given a SELECT-only query load, I'd expect PG to reach a state of zero new writes fairly quickly (at the latest, after a vacuum and checkpoint have occurred). The real problem is that any single update operation will generate quite a number of disk writes, the more so the smaller your shared_buffers setting :-(. It's not at all optimized to minimize writes in a low-but-not-zero-update-traffic situation. So you'd likely be facing some issues with FLASH lifetime. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Small OS ports Handheld devices
Simon Riggs wrote: Is there a definitive list of supported ports? (for anything) http://www.postgresql.org/docs/7.4/static/supported-platforms.html For example the Linux/arm port can run on many PDA's but because of the write intensity and the limited write cycles of flash memory this continues to be a toy setup. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Small OS ports Handheld devices
Simon Riggs wrote: Is there a definitive list of supported ports? (for anything) http://www.postgresql.org/docs/7.4/static/supported-platforms.html For example the Linux/arm port can run on many PDA's but because of the write intensity and the limited write cycles of flash memory this continues to be a toy setup. Why? If you really need PostgreSQL on your PDA don't use Flash. Use an IDE drive. http://www.flash-memory-store.com/ibmmicrodrives.html http://www.memorysuppliers.com/ibmmic1gbwtr.html $369 for 4GB of storage in a compact flash card is not all that bad. $189 for 2.2GB is very reasonable when you consider the 512MB CF is going for $149! These are basically IDE drives in a compact flash form factor, as such they should have lifetimes approaching a regular IDE drive, right? Just don't bounce them around too much. Later Rob ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS]
=?gb2312?B?emh1YW5namlmZW5n?= [EMAIL PROTECTED] writes: PGh0bWw+PHByZT48UFJFPmhlbGxvLDxCUj5ldmVyeW9uZSE8QlI+Jm5ic3A7 SSB3YW50IHRvIGtub3cgaG93IHBvc3RncmVzcWwgcmVtYWluIGl0cyB1bmlx dWUgaW5kZXggdW5pcXVlIGFmdGVyIDxCUj5pIGhhdmUgY2hhbmdlIGl0LkZv ciBleGFtcGxlOjxCUj4mbmJzcDtjcmVhdGUgdGFibGUgaHVtYW4oYWdlIGlu dCxuYW1lIGNoYXIoMikpOzxCUj50aGVuIGluIHRoZSBwZ19hdHRyaWJ1dGUg Y2F0YWxvZyB0aGUgdW5pcXVlIGluZGV4IHBnX2F0dHJpYnV0ZV9yZWxpZF9h dHRudW1faW5kZXggd2lsbDxCUj51c2UgcmVsaWQgb2YgJ2h1bWFuJ2FuZCBh dHRudW0gb2YgYWdlIHRvIGNyZWF0ZSBhIGluZGV4IHR1cGxlIGZvciBhdHRy aWJ1dGU6YWdlLHdoaWNoIGlzIGEgdW5pcXVlIGluZGV4IHR1cGxlIGZvciBh IHR1cGxlIHdoaWNoIGRlc2NyaWJlIGFnZSBpbiBwZ19hdHRyaWJ1dGUsYW5k IGkgZG86PEJSPiZuYnNwO2FsdGVyIHRhYmxlIGh1bWFuIHJlbmFtZSBhZ2Ug dG8gYmlydGhkYXRlOzxCUj50aGVuIGluIHRoZSB1bmlxdWUgaW5kZXggcGdf YXR0cmlidXRlX3JlbGlkX2F0dG51bV9pbmRleCB3aWxsIGFwcGVhciB0d28g aW5kZXggdHVwbGUgPEJSPndoaWNoIGtleXMgYXJlIHJlbGlkIG9mICdodW1h bicgYW5kIGF0dG51bToxIHRvIGRlc2NyaWJlIGFnZSAsYnV0IHRoZXJlIGlz IG5vIGVycm9yIGhhcHBlbmluZyEhISFTbyBpIGFtIGludGVyZXN0ZWQgaW4g dGhlIG1ldGhvZCBvZiByZW1haW4gdGhlIGR5bmFtaWMgaW5kZXggdmVyeSBt dWNoLjxCUj4mbmJzcDt3b3VsZCB5b3UgcGxlYXNlIHRlbGwgaG93IHBvc3Rn cmVzIGRvIGl0IGFuZCB0aGUgc3JjIG9mIHRoaXM/IDxCUj50aGFua3MgOikg PC9QUkU+PFBSRT4mbmJzcDs8L1BSRT48UFJFPqGhoaGhoaGhoaGhoaGhoaHW wjxCUj7A8aOhPEJSPiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOzwv UFJFPjxQUkU+oaGhoaGhoaGhoaGhoaGhodevvMy35jxCUj6hoaGhoaGhoaGh oaGhoaGhPEEgaHJlZj0ibWFpbHRvOnpodWFuZ2ppZmVuZ0AyNjMubmV0Ij56 aHVhbmdqaWZlbmdAMjYzLm5ldDwvQT48QlI+oaGhoaGhoaGhoaGhoaGhoaGh oaEyMDA0LTA0LTI5PC9QUkU+PC9wcmU+PC9odG1sPg0KDQoNCg0KDQoNCj09 PT09PT09PT09PT09PT09PT09PT09PT09DQoyNjO159fT08q8/qOt0MXAtdPK 19TXqNK1 I'd have answered this if my editor could quote it reasonably, but I don't have the energy to deal with the above... In future, please post to pghackers in plain text (lose the HTML, too) and include a relevant Subject: line. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Call for 7.5 feature completion
On Thu, 29 Apr 2004, Tom Lane wrote: Marc G. Fournier [EMAIL PROTECTED] writes: *If* June 1st comes along, and Win32 isn't ready, there is nothing wrong with freezing the code *except* for a pending Win32 patch ... Yeah there is ... In the first place it's unfair to other developers to make schedule slips at the last moment, and especially to *plan* to do so. Isn't it equally unfair to slip the scheduale that developers that have been working on some large features (PITR, 2PC immediately coming to mind) have been working towards based on a deadline? If Win32 that much more important then those other features? I would prefer we determine at least two weeks, preferably a month, in advance whether there is going to be a schedule change. I think it's entirely appropriate to decide now whether we are going to hold to June 1 feature freeze or slip it. That gives other people a chance to do useful work while we approach the deadline. In the second, any Win32 issues that remain unresolved as of June 1 are likely to be major enough (in terms of code impact) that we can't realistically contemplate slipping in post-beta patches for them. The fallback position is going to have to be no (supported) Win32 port in 7.5, not we'll fix this in beta. Personally, I think there are alot of large features that ppl have been hard at getting complete in time for June 1st that we should stick to it, else we're going to end up with 'yet another release' delayed in hopes that the outstanding bugs in Win32 will get fixed in a reasonable amount of time ... June 1st, let's do beta for 7.5 and then branch onto 8.0, with 8.0 key'd to the Win32 Native port being finished ... If that means 8.0 happens to be September 1st, so be it ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Number of pages in a random sample
On Mon, 26 Apr 2004 08:08:16 -0700, Sailesh Krishnamurthy [EMAIL PROTECTED] wrote: A Bi-Level Bernoulli Scheme for Database Sampling Peter Haas, Christian Koenig (SIGMOD 2004) Does this apply to our problem? AFAIK with Bernoulli sampling you don't know the sample size in advance. Anyway, thanks for the hint. Unfortunately I couldn't find the document. Do you have a link? Servus Manfred ---(end of broadcast)--- TIP 8: explain analyze is your friend
