Re: [HACKERS] CVS corruption/mistagging?
Tom Lane wrote: In the meantime, though, it's not quite clear why this would lead to a buildfarm failure --- it should just mean a lot of extraneous files appearing in a fresh checkout. (Looks a bit harder ... Oh, it looks like btree_gist has some files that used to be autogenerated and are now in CVS, so the bogusly new versions from CVS are suppressing the desired generation from the old btree_num.c file.) Funny enoughy, that's exactly the problem I saw on win32 when I rsynced from a windows box and tried a local CVS checkout :-) Once I rsynced to a local linux instead and checked out from there, things worked again :) //Magnus ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] CVS corruption/mistagging?
Tom Lane wrote: So I think that cvs rtag -d REL7_4_STABLE pgsql will fix it. I'd like someone to double-check that though. Also maybe we should back up the repository first? Just for your info: all VMs on tribble, which includes cvs, are backed up at 02:30 every day, CEST (that's 00:30 UTC right now, but it follows the local time on tribble so it'll be 23:30 UTC when austra switches later this year. Perhaps we should switch the machine to GMT, but that's a different discussion) For something like this it's good to run a special backup before that of course, but it's probably good to know when the backups run anyway... //Magnus ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] CVS corruption/mistagging?
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: So I'd say your strategy looks good - backup and remove the phony tag. I'd also say we should probably be logging tag commands in taginfo. Presumably we mere mortal committers should not be doing any tagging whatsoever, and tags should only be applied in preparation for releases. +1 ... we should at least log such commands, and maybe disallow to anyone except Marc's pgsql account. Particularly since they don't get reported in pgsql-committers messages (or is that something we can/should change?) I think you can, by sticking a command in the taginfo command. It's the same script that Marc is using to try to restrict who can do it - once he has that working, it should be as simple as adding a mail command to the end of that script. And +1 on doing it. //Magnus ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] CVS corruption/mistagging?
Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: I'd like someone to double-check that though. Also maybe we should back up the repository first? Just for your info: all VMs on tribble, which includes cvs, are backed up at 02:30 every day, CEST Good, but the salient followup questions to that are (1) backed up to where exactly?, and (2) how many days' past backups could we get at, if we had to? regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] CVS corruption/mistagging?
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: I'd like someone to double-check that though. Also maybe we should back up the repository first? Just for your info: all VMs on tribble, which includes cvs, are backed up at 02:30 every day, CEST Good, but the salient followup questions to that are (1) backed up to where exactly?, and (2) how many days' past backups could we get at, if we had to? 1) The entire VM, with dump 2) We have today, yesterday and one weekly (copies the daily on monday, if I read it right) They are dumped to a NFS share on this schedule. That NFS share is dumped to tape by systems at Conova - I'll let Stefan fill in the details about that. //Magnus ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] CVS corruption/mistagging?
Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Good, but the salient followup questions to that are (1) backed up to where exactly?, and (2) how many days' past backups could we get at, if we had to? They are dumped to a NFS share on this schedule. That NFS share is dumped to tape by systems at Conova - I'll let Stefan fill in the details about that. That's good as far as it goes, but seeing that PG is a worldwide organization now, I wonder whether our primary CVS shouldn't have backups on several continents. Pardon my paranoia ... but our collective arses have been saved by offsite backups at least once already ... regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] CVS corruption/mistagging?
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Good, but the salient followup questions to that are (1) backed up to where exactly?, and (2) how many days' past backups could we get at, if we had to? They are dumped to a NFS share on this schedule. That NFS share is dumped to tape by systems at Conova - I'll let Stefan fill in the details about that. That's good as far as it goes, but seeing that PG is a worldwide organization now, I wonder whether our primary CVS shouldn't have backups on several continents. Pardon my paranoia ... but our collective arses have been saved by offsite backups at least once already ... The CVS repository is already synced almost-live back to both svr1 in panama and the anoncvs box in the US. It will be synced to the new VM marc is setting up as well. I assume these machines also have some kind of backup - Marc will have to fill in on that. Oh, and the plan is to have the entire VM synced up between austria and panama machine once / day again once Marc has fixed the issues with that box. //Magnus ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] CVS corruption/mistagging?
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Good, but the salient followup questions to that are (1) backed up to where exactly?, and (2) how many days' past backups could we get at, if we had to? They are dumped to a NFS share on this schedule. That NFS share is dumped to tape by systems at Conova - I'll let Stefan fill in the details about that. That's good as far as it goes, but seeing that PG is a worldwide organization now, I wonder whether our primary CVS shouldn't have backups on several continents. Pardon my paranoia ... but our collective arses have been saved by offsite backups at least once already ... We already have that -- there's plenty of people who have rsync'ed copies of the repository. Now, I don't back it up, so if someone makes a mistake my copy will faithfully follow it. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] CVS corruption/mistagging?
On 8/15/07, Tom Lane [EMAIL PROTECTED] wrote: I wrote: Kris Jurka [EMAIL PROTECTED] writes: It looks like parts of the CVS repository have been mistagged as belonging to REL7_4_STABLE or have been corrupted somehow: I have no idea how you make CVS do that, but I'm sure there is some magic one-liner for it. Something like cvs tag without -F on the HEAD? -- marko ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] CVS corruption/mistagging?
Magnus Hagander wrote: Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Good, but the salient followup questions to that are (1) backed up to where exactly?, and (2) how many days' past backups could we get at, if we had to? They are dumped to a NFS share on this schedule. That NFS share is dumped to tape by systems at Conova - I'll let Stefan fill in the details about that. That's good as far as it goes, but seeing that PG is a worldwide organization now, I wonder whether our primary CVS shouldn't have backups on several continents. Pardon my paranoia ... but our collective arses have been saved by offsite backups at least once already ... The CVS repository is already synced almost-live back to both svr1 in panama and the anoncvs box in the US. It will be synced to the new VM marc is setting up as well. I assume these machines also have some kind of backup - Marc will have to fill in on that. Oh, and the plan is to have the entire VM synced up between austria and panama machine once / day again once Marc has fixed the issues with that box. yeah that has been my understanding as well though I'm not sure if marc has set up the sync of the full VM already. Stefan ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] CVS corruption/mistagging?
Magnus Hagander wrote: Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: I'd like someone to double-check that though. Also maybe we should back up the repository first? Just for your info: all VMs on tribble, which includes cvs, are backed up at 02:30 every day, CEST Good, but the salient followup questions to that are (1) backed up to where exactly?, and (2) how many days' past backups could we get at, if we had to? 1) The entire VM, with dump 2) We have today, yesterday and one weekly (copies the daily on monday, if I read it right) They are dumped to a NFS share on this schedule. That NFS share is dumped to tape by systems at Conova - I'll let Stefan fill in the details about that. that's exactly what happens on the backup side here - but maybe I missed an earlier mail - is something broken on tribble or on the new CVS VM that requires a restore from backup ? Stefan ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] CVS corruption/mistagging?
Stefan Kaltenbrunner wrote: Magnus Hagander wrote: Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: I'd like someone to double-check that though. Also maybe we should back up the repository first? Just for your info: all VMs on tribble, which includes cvs, are backed up at 02:30 every day, CEST Good, but the salient followup questions to that are (1) backed up to where exactly?, and (2) how many days' past backups could we get at, if we had to? 1) The entire VM, with dump 2) We have today, yesterday and one weekly (copies the daily on monday, if I read it right) They are dumped to a NFS share on this schedule. That NFS share is dumped to tape by systems at Conova - I'll let Stefan fill in the details about that. that's exactly what happens on the backup side here - but maybe I missed an earlier mail - is something broken on tribble or on the new CVS VM that requires a restore from backup ? erm -ENOTENOUGHCOFFEE Stefan ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] CVS corruption/mistagging?
Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: we should at least log such commands, and maybe disallow to anyone except Marc's pgsql account. I don't think we should disallow it. Or otherwise we might one day be stuck if we need to release while some specific person is on vacation. I never understood why tagging uses a special account anyway. It should be done as the person doing the tagging. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] CVS corruption/mistagging?
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Good, but the salient followup questions to that are (1) backed up to where exactly?, and (2) how many days' past backups could we get at, if we had to? They are dumped to a NFS share on this schedule. That NFS share is dumped to tape by systems at Conova - I'll let Stefan fill in the details about that. That's good as far as it goes, but seeing that PG is a worldwide organization now, I wonder whether our primary CVS shouldn't have backups on several continents. Pardon my paranoia ... but our collective arses have been saved by offsite backups at least once already ... Yes, I think we could improve on that. Have we considered more sophisticated solutions that provide incremental backup on a more frequent basis? I'd be inclined to use Bacula or similar (and it uses Postgres for its metadata store :-) ). Ideally I think we'd like to be able fairly easily and quickly to roll the repo (or some portion of it) back to a fairly arbitrary and fairly precise (say within an hour or two) point in recent time. Meanwhile, those of us who rsync the entire repo could easily make rolling backup copies. Arguably this might be better done from a repo made using --numeric-ids. Tarred and compressed it's a shade under 90 Mb, which isn't huge. If people do this at various times of the day we'd get pretty good coverage :-) cheers andrew ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] CVS corruption/mistagging?
Andrew Dunstan wrote: Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Good, but the salient followup questions to that are (1) backed up to where exactly?, and (2) how many days' past backups could we get at, if we had to? They are dumped to a NFS share on this schedule. That NFS share is dumped to tape by systems at Conova - I'll let Stefan fill in the details about that. That's good as far as it goes, but seeing that PG is a worldwide organization now, I wonder whether our primary CVS shouldn't have backups on several continents. Pardon my paranoia ... but our collective arses have been saved by offsite backups at least once already ... Yes, I think we could improve on that. Have we considered more sophisticated solutions that provide incremental backup on a more frequent basis? I'd be inclined to use Bacula or similar (and it uses Postgres for its metadata store :-) ). Ideally I think we'd like to be able fairly easily and quickly to roll the repo (or some portion of it) back to a fairly arbitrary and fairly precise (say within an hour or two) point in recent time. well yeah - while I do think that something as complex like bacula is probably overkill for our needs we can certainly improve over the current state. One thing to consider is that we actually have two major scenarios to deal with: 1. simple repo corruption (accident,cvs software bug, admin error) this one might require us to restore the repo from an older backup in the case the corruption cannot be repaired easily. For this we already have myriads of copies of the trees out in the wild but i might be a good idea to keep a number snapshots of the main repo on the CVS-VPS itself (either done every few hours or made as part of the push to anoncvs and svr1). This way we could do a very simple inplace recovery on the same running VPS instance with fairly low inpact to all the commiters (and depending parts of teh infrastructure) 2. total loss of the main CVS-VPS (extended power failure, hardware error, OS bug, admin error, fire, some other catastropic event) - in this case we will have to fail over to one of the other project datacenters and for this we need to have full regular copies of the whole VM on external hosts. Stefan ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] CVS corruption/mistagging?
Peter Eisentraut wrote: Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: we should at least log such commands, and maybe disallow to anyone except Marc's pgsql account. I don't think we should disallow it. Or otherwise we might one day be stuck if we need to release while some specific person is on vacation. Is this really a problem in practise ? If such a need arises any commiter (or sysadmin) would probably be able to remove that restriction in a few minutes. I think the point here is to prevent such things done by accident(vs. on purpose) which a script like that seems like a fairly simple yet effective solution. Stefan ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] CVS corruption/mistagging?
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Peter Eisentraut wrote: Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: we should at least log such commands, and maybe disallow to anyone except Marc's pgsql account. I don't think we should disallow it. Or otherwise we might one day be stuck if we need to release while some specific person is on vacation. Is this really a problem in practise ? I'd be happy if such commands got reported to the pgsql-committers mailing list. The real problem with this mistake is not so much that it was made as that we had to do detective work to find out. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] CVS corruption/mistagging?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Wednesday, August 15, 2007 12:11:35 +0200 Peter Eisentraut [EMAIL PROTECTED] wrote: Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: we should at least log such commands, and maybe disallow to anyone except Marc's pgsql account. I don't think we should disallow it. Or otherwise we might one day be stuck if we need to release while some specific person is on vacation. This isn't a big issue ... note that the 'restriction' is easy to remove ... you checkout CVSROOT, modify taginfo and comment out the ALL l ine and check that in ... What this will prevent is an 'accidental tagging' ... you would have to consciously remove the restriction .. but its something anyone could do ... I never understood why tagging uses a special account anyway. It should be done as the person doing the tagging. Agreed, I'm going to start doing it as myself from now on ... I'm not even 100% certain *why* I started doing it as pgsql in the first place ... - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGwye54QvfyHIvDvMRAiHEAKCHGtXA+r5PM1SoBewMJDo3An7BKACg0//z gM7eZNoTEU4sqwBIHprFK1k= =/Qga -END PGP SIGNATURE- ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] CVS corruption/mistagging?
Marc G. Fournier wrote: --On Wednesday, August 15, 2007 12:11:35 +0200 Peter Eisentraut [EMAIL PROTECTED] wrote: Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: we should at least log such commands, and maybe disallow to anyone except Marc's pgsql account. I don't think we should disallow it. Or otherwise we might one day be stuck if we need to release while some specific person is on vacation. This isn't a big issue ... note that the 'restriction' is easy to remove ... you checkout CVSROOT, modify taginfo and comment out the ALL l ine and check that in ... What this will prevent is an 'accidental tagging' ... you would have to consciously remove the restriction .. but its something anyone could do ... I never understood why tagging uses a special account anyway. It should be done as the person doing the tagging. Agreed, I'm going to start doing it as myself from now on ... I'm not even 100% certain *why* I started doing it as pgsql in the first place ... If you're doing that, we should probably just delete the pgsql userid from the system? Or at least change it so it doesn't have 'dev' permissions. That way you can't do it wrong in that direction ;-) Seems reasonable? //Magnus ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] CVS corruption/mistagging?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Wednesday, August 15, 2007 20:57:14 +0200 Magnus Hagander [EMAIL PROTECTED] wrote: Marc G. Fournier wrote: --On Wednesday, August 15, 2007 12:11:35 +0200 Peter Eisentraut [EMAIL PROTECTED] wrote: Am Mittwoch, 15. August 2007 04:20 schrieb Tom Lane: we should at least log such commands, and maybe disallow to anyone except Marc's pgsql account. I don't think we should disallow it. Or otherwise we might one day be stuck if we need to release while some specific person is on vacation. This isn't a big issue ... note that the 'restriction' is easy to remove ... you checkout CVSROOT, modify taginfo and comment out the ALL l ine and check that in ... What this will prevent is an 'accidental tagging' ... you would have to consciously remove the restriction .. but its something anyone could do ... I never understood why tagging uses a special account anyway. It should be done as the person doing the tagging. Agreed, I'm going to start doing it as myself from now on ... I'm not even 100% certain *why* I started doing it as pgsql in the first place ... If you're doing that, we should probably just delete the pgsql userid from the system? Or at least change it so it doesn't have 'dev' permissions. That way you can't do it wrong in that direction ;-) Seems reasonable? there is no pgsql user *on* cvs.postgresql.org, at least there wasn't when I logged on last night: cvs# grep pgsql /etc/passwd cvs# - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGw1AR4QvfyHIvDvMRArsSAJ9WzXyMu7Io6dP8kDnR5HHex1f1gQCg7iDq 3p2RrAzww4dCDPVCyMozAnM= =L+/M -END PGP SIGNATURE- ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] CVS corruption/mistagging?
Marc G. Fournier wrote: If you're doing that, we should probably just delete the pgsql userid from the system? Or at least change it so it doesn't have 'dev' permissions. That way you can't do it wrong in that direction ;-) Seems reasonable? there is no pgsql user *on* cvs.postgresql.org, at least there wasn't when I logged on last night: cvs# grep pgsql /etc/passwd cvs# Ah, seems I was faster than I expected myself :-) Problem solved then. //Magnus ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] CVS corruption/mistagging?
Kris Jurka [EMAIL PROTECTED] writes: It looks like parts of the CVS repository have been mistagged as belonging to REL7_4_STABLE or have been corrupted somehow: http://developer.postgresql.org/cvsweb.cgi/pgsql/contrib/btree_gist/btree_bit.c?sortby=date;only_with_tag=REL7_4_STABLE Hmm ... btree_bit.c shouldn't be in 7.4 at all. I can't help thinking this has something to do with the recent CVS server move. Magnus, any thoughts? regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] CVS corruption/mistagging?
I wrote: Kris Jurka [EMAIL PROTECTED] writes: It looks like parts of the CVS repository have been mistagged as belonging to REL7_4_STABLE or have been corrupted somehow: http://developer.postgresql.org/cvsweb.cgi/pgsql/contrib/btree_gist/btree_bit.c?sortby=date;only_with_tag=REL7_4_STABLE Hmm ... btree_bit.c shouldn't be in 7.4 at all. I did a fresh checkout of the 7.4 branch and diff'd against my local copy, and it seems clear that every file that was not in 7.4 at all has had its HEAD version tagged as REL7_4_STABLE. The files that did exist then are all right. That's throughout the whole tree, not just in contrib/btree_gist. I have no idea how you make CVS do that, but I'm sure there is some magic one-liner for it. Checking the files in contrib/btree_gist on the CVS server gives a pretty good fix on who changed it and when: ls -la total 466 drwxrwxr-x 6 scrappy dev 1024 Aug 14 22:30 . drwxrwxr-x 90 scrappy dev 2048 Aug 14 22:27 .. drwxrwxr-x 2 scrappy dev512 Apr 20 15:16 Attic -r--r--r-- 1 tgl dev 9555 Jun 26 22:05 Makefile,v -r--r--r-- 1 258 dev 5870 Apr 20 16:19 README.btree_gist,v -r--r--r-- 1 meskes dev 12052 Aug 12 09:50 btree_bit.c,v -r--r--r-- 1 meskes dev 9921 Aug 12 09:50 btree_bytea.c,v -r--r--r-- 1 meskes dev 10246 Aug 12 09:50 btree_cash.c,v -r--r--r-- 1 meskes dev 11433 Aug 12 09:50 btree_date.c,v -r--r--r-- 1 meskes dev 10230 Aug 12 09:50 btree_float4.c,v -r--r--r-- 1 meskes dev 10093 Aug 12 09:50 btree_float8.c,v -r--r--r-- 1 meskes dev 33935 Aug 12 09:50 btree_gist.c,v -r--r--r-- 1 258 dev 4325 Apr 20 15:16 btree_gist.h,v -r--r--r-- 1 258 dev 58744 Apr 20 16:19 btree_gist.sql.in,v -r--r--r-- 1 meskes dev 14736 Aug 12 09:50 btree_inet.c,v -r--r--r-- 1 meskes dev 10253 Aug 12 09:50 btree_int2.c,v -r--r--r-- 1 meskes dev 10240 Aug 12 09:50 btree_int4.c,v -r--r--r-- 1 meskes dev 10254 Aug 12 09:50 btree_int8.c,v -r--r--r-- 1 meskes dev 18111 Aug 12 09:50 btree_interval.c,v -r--r--r-- 1 meskes dev 12025 Aug 12 09:50 btree_macaddr.c,v -r--r--r-- 1 meskes dev 14671 Aug 12 09:50 btree_numeric.c,v -r--r--r-- 1 meskes dev 9796 Aug 12 09:50 btree_oid.c,v -r--r--r-- 1 meskes dev 18247 Aug 12 09:50 btree_text.c,v -r--r--r-- 1 meskes dev 26180 Aug 12 09:50 btree_time.c,v -r--r--r-- 1 258 dev 29712 Apr 20 15:16 btree_ts.c,v -r--r--r-- 1 meskes dev 17588 Aug 12 09:50 btree_utils_num.c,v -r--r--r-- 1 meskes dev 8959 Aug 12 09:50 btree_utils_num.h,v -r--r--r-- 1 meskes dev 48824 Aug 12 09:50 btree_utils_var.c,v -r--r--r-- 1 meskes dev 7099 Aug 12 09:50 btree_utils_var.h,v drwxrwxr-x 3 scrappy dev 1024 Aug 14 22:27 data drwxrwxr-x 3 scrappy dev 1024 Aug 14 22:27 expected drwxrwxr-x 3 scrappy dev 1024 Aug 14 22:27 sql -r--r--r-- 1 meskes dev 10021 Aug 12 09:50 uninstall_btree_gist.sql,v Michael, want to fess up to what you did? In the meantime, though, it's not quite clear why this would lead to a buildfarm failure --- it should just mean a lot of extraneous files appearing in a fresh checkout. (Looks a bit harder ... Oh, it looks like btree_gist has some files that used to be autogenerated and are now in CVS, so the bogusly new versions from CVS are suppressing the desired generation from the old btree_num.c file.) regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] CVS corruption/mistagging?
Tom Lane wrote: In the meantime, though, it's not quite clear why this would lead to a buildfarm failure --- it should just mean a lot of extraneous files appearing in a fresh checkout. (Looks a bit harder ... Oh, it looks like btree_gist has some files that used to be autogenerated and are now in CVS, so the bogusly new versions from CVS are suppressing the desired generation from the old btree_num.c file.) Looking at the Committers mail, it looks like there have only been two very small commits since Michael's series of commits around 12 to 13.5 hours ago, and before that nothing since around 28 hours ago. Do we have a backup snapshot of the repo taken during that time that we can roll back to? That might be simpler than doing surgery on the repo. cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] CVS corruption/mistagging?
Andrew Dunstan [EMAIL PROTECTED] writes: Looking at the Committers mail, it looks like there have only been two very small commits since Michael's series of commits around 12 to 13.5 hours ago, and before that nothing since around 28 hours ago. Do we have a backup snapshot of the repo taken during that time that we can roll back to? That might be simpler than doing surgery on the repo. It looks to me like the mistake was unrelated to any commit, and was made at Aug 12 09:50 UTC. (Aside from the file-timestamp evidence, caracara's build failure at 2007-08-12 213845 shows that the problem is more than 2 days old.) I count seven commits of my own and several of Michael's since then. If I'm right that a simple cvs rtag will fix it, I'd rather do that than try to reapply patches. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] CVS corruption/mistagging?
I wrote: I did a fresh checkout of the 7.4 branch and diff'd against my local copy, and it seems clear that every file that was not in 7.4 at all has had its HEAD version tagged as REL7_4_STABLE. The files that did exist then are all right. That's throughout the whole tree, not just in contrib/btree_gist. As for getting out of it: it looks like the misapplied tags are revision tags not branch tags, eg RCS file: /cvsroot/pgsql/contrib/btree_gist/btree_bit.c,v Working file: btree_bit.c head: 1.7 branch: locks: strict access list: symbolic names: REL7_4_STABLE: 1.7 --- wrong REL8_0_13: 1.2 REL8_1_9: 1.6 REL8_2_4: 1.6 REL8_0_12: 1.2 REL8_1_8: 1.6 REL8_2_3: 1.6 REL8_0_11: 1.2 REL8_1_7: 1.6 REL8_2_2: 1.6 REL8_0_10: 1.2 REL8_1_6: 1.6 REL8_2_1: 1.6 REL8_2_STABLE: 1.6.0.4 which I think means that there's a one-liner way out of it. The CVS manual quoth: To delete a tag, specify the `-d' option to either `cvs tag' or `cvs rtag'. For example: cvs rtag -d rel-0-4 tc deletes the non-branch tag `rel-0-4' from the module `tc'. In the event that branch tags are encountered within the repository with the given name, a warning message will be issued and the branch tag will not be deleted. If you are absolutely certain you know what you are doing, the `-B' option may be specified to allow deletion of branch tags. In that case, any non-branch tags encountered will trigger warnings and will not be deleted. So I think that cvs rtag -d REL7_4_STABLE pgsql will fix it. I'd like someone to double-check that though. Also maybe we should back up the repository first? regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] CVS corruption/mistagging?
Tom Lane wrote: So I think that cvs rtag -d REL7_4_STABLE pgsql will fix it. I'd like someone to double-check that though. I will test on a copy of my mirror. Also maybe we should back up the repository first? Amen. cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] CVS corruption/mistagging?
Andrew Dunstan wrote: Tom Lane wrote: So I think that cvs rtag -d REL7_4_STABLE pgsql will fix it. I'd like someone to double-check that though. I will test on a copy of my mirror. I copied the mirror, did a checkout from it, ran the command above in the checked out version, then remove my checkout and checked out REL7_4_STABLE and did a diff against my copy from the postgres repo that hasn't been updated for some time. There were no significant differences, and no extra files. So I'd say your strategy looks good - backup and remove the phony tag. I'd also say we should probably be logging tag commands in taginfo. Presumably we mere mortal committers should not be doing any tagging whatsoever, and tags should only be applied in preparation for releases. Anyway, if we actually logged it we'd catch anything untoward early rather than possibly days later. cheers andrew ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] CVS corruption/mistagging?
Andrew Dunstan [EMAIL PROTECTED] writes: So I'd say your strategy looks good - backup and remove the phony tag. I'd also say we should probably be logging tag commands in taginfo. Presumably we mere mortal committers should not be doing any tagging whatsoever, and tags should only be applied in preparation for releases. +1 ... we should at least log such commands, and maybe disallow to anyone except Marc's pgsql account. Particularly since they don't get reported in pgsql-committers messages (or is that something we can/should change?) Meanwhile, is there anyone around who can either (1) tar up the repository directory tree as root, or (2) confirm that a tarball made by a non-root committer is sufficient? regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] CVS corruption/mistagging?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Tuesday, August 14, 2007 22:20:16 -0400 Tom Lane [EMAIL PROTECTED] wrote: +1 ... we should at least log such commands, and maybe disallow to anyone except Marc's pgsql account. Particularly since they don't get reported in pgsql-committers messages (or is that something we can/should change?) Meanwhile, is there anyone around who can either (1) tar up the repository directory tree as root, or (2) confirm that a tarball made by a non-root committer is sufficient? I've just finished tar'ng up the cvs repository as root ... I believe one as a non-root would be sufficient also, since it *should* retain file ownership / permissions with the -p option ... cvs# ls -lt cvsroot.aug14_2007.tar.bz2 - -rw-r--r-- 1 root wheel 62555651 Aug 15 02:52 cvsroot.aug14_2007.tar.bz2 cvs# du -sk cvsroot 414406 cvsroot - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGwmq14QvfyHIvDvMRAsyFAJ9AND3JTDmOMR48Hl3Fdse2lVCjSwCglsu1 Pxj1LPPPx311rpY2XFj+Kvo= =Gz6E -END PGP SIGNATURE- ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] CVS corruption/mistagging?
Marc G. Fournier [EMAIL PROTECTED] writes: On Tuesday, August 14, 2007 22:20:16 -0400 Tom Lane [EMAIL PROTECTED] wrote: Meanwhile, is there anyone around who can either (1) tar up the repository directory tree as root, or (2) confirm that a tarball made by a non-root committer is sufficient? I've just finished tar'ng up the cvs repository as root ... Great --- launching cvs rtag command now. I believe one as a non-root would be sufficient also, since it *should* retain file ownership / permissions with the -p option ... That's what I thought too, but a little extra paranoia doesn't hurt. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] CVS corruption/mistagging?
I wrote: Great --- launching cvs rtag command now. Done, and I got a plausible-looking mix of messages like cvs rtag: Not removing branch tag `REL7_4_STABLE' from `/cvsroot/pgsql/src/tutorial/funcs_new.c,v'. and a fresh checkout of REL7_4_STABLE now matches what I had locally. So I think we walked away from that one. TODO items: * restrict, or at least log, cvs tag/rtag commands. Maybe report them to pgsql-committers. * I notice that the repository contains some files owned by no-longer-existent accounts, eg 258 in contrib/btree_gist: ls -la total 460 drwxrwxr-x 6 scrappy dev 1024 Aug 15 03:00 . drwxrwxr-x 90 scrappy dev 2048 Aug 15 03:00 .. drwxrwxr-x 2 scrappy dev512 Apr 20 15:16 Attic -r--r--r-- 1 tgl dev 9555 Jun 26 22:05 Makefile,v -r--r--r-- 1 258 dev 5870 Apr 20 16:19 README.btree_gist,v -r--r--r-- 1 tgl dev 12033 Aug 15 03:00 btree_bit.c,v -r--r--r-- 1 tgl dev 9902 Aug 15 03:00 btree_bytea.c,v -r--r--r-- 1 tgl dev 10227 Aug 15 03:00 btree_cash.c,v -r--r--r-- 1 tgl dev 11414 Aug 15 03:00 btree_date.c,v -r--r--r-- 1 tgl dev 10211 Aug 15 03:00 btree_float4.c,v -r--r--r-- 1 tgl dev 10074 Aug 15 03:00 btree_float8.c,v -r--r--r-- 1 tgl dev 33915 Aug 15 03:00 btree_gist.c,v -r--r--r-- 1 258 dev 4325 Apr 20 15:16 btree_gist.h,v -r--r--r-- 1 258 dev 58744 Apr 20 16:19 btree_gist.sql.in,v -r--r--r-- 1 tgl dev 14717 Aug 15 03:00 btree_inet.c,v -r--r--r-- 1 tgl dev 10234 Aug 15 03:00 btree_int2.c,v -r--r--r-- 1 tgl dev 10221 Aug 15 03:00 btree_int4.c,v -r--r--r-- 1 tgl dev 10235 Aug 15 03:00 btree_int8.c,v I suppose this is due to Magnus' cleanup of unused committers accounts. That was a good thing, but for security's sake these files ought to be chown'd to some existing committer's account. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] CVS corruption/mistagging?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Tuesday, August 14, 2007 23:26:03 -0400 Tom Lane [EMAIL PROTECTED] wrote: I wrote: Great --- launching cvs rtag command now. Done, and I got a plausible-looking mix of messages like cvs rtag: Not removing branch tag `REL7_4_STABLE' from `/cvsroot/pgsql/src/tutorial/funcs_new.c,v'. and a fresh checkout of REL7_4_STABLE now matches what I had locally. So I think we walked away from that one. TODO items: * restrict, or at least log, cvs tag/rtag commands. Maybe report them to pgsql-committers. working on it ... have code for it, I've just spent so much time with CVSROOT = /cvsroot, I have to relearn remote CVS :) ls -la total 460 drwxrwxr-x 6 scrappy dev 1024 Aug 15 03:00 . drwxrwxr-x 90 scrappy dev 2048 Aug 15 03:00 .. drwxrwxr-x 2 scrappy dev512 Apr 20 15:16 Attic -r--r--r-- 1 tgl dev 9555 Jun 26 22:05 Makefile,v -r--r--r-- 1 258 dev 5870 Apr 20 16:19 README.btree_gist,v -r--r--r-- 1 tgl dev 12033 Aug 15 03:00 btree_bit.c,v -r--r--r-- 1 tgl dev 9902 Aug 15 03:00 btree_bytea.c,v -r--r--r-- 1 tgl dev 10227 Aug 15 03:00 btree_cash.c,v -r--r--r-- 1 tgl dev 11414 Aug 15 03:00 btree_date.c,v -r--r--r-- 1 tgl dev 10211 Aug 15 03:00 btree_float4.c,v -r--r--r-- 1 tgl dev 10074 Aug 15 03:00 btree_float8.c,v -r--r--r-- 1 tgl dev 33915 Aug 15 03:00 btree_gist.c,v -r--r--r-- 1 258 dev 4325 Apr 20 15:16 btree_gist.h,v -r--r--r-- 1 258 dev 58744 Apr 20 16:19 btree_gist.sql.in,v -r--r--r-- 1 tgl dev 14717 Aug 15 03:00 btree_inet.c,v -r--r--r-- 1 tgl dev 10234 Aug 15 03:00 btree_int2.c,v -r--r--r-- 1 tgl dev 10221 Aug 15 03:00 btree_int4.c,v -r--r--r-- 1 tgl dev 10235 Aug 15 03:00 btree_int8.c,v I suppose this is due to Magnus' cleanup of unused committers accounts. That was a good thing, but for security's sake these files ought to be chown'd to some existing committer's account. I can do a quick chown -R scrappy on the whole repository ... ok? - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGwnf34QvfyHIvDvMRAnGTAJ43gELVZ5Q5SpV/jppR9rODB/ILaACeKCEN DE16fnY/PmKlrvdHFaJeIKI= =VCFu -END PGP SIGNATURE- ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] CVS corruption/mistagging?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Tuesday, August 14, 2007 23:26:03 -0400 Tom Lane [EMAIL PROTECTED] wrote: * restrict, or at least log, cvs tag/rtag commands. Maybe report them to pgsql-committers. It should be done ... if you try and create a tag, it should generate an error message ... not sure of the best way to test that it actually works, since, if it doesn't work, of course, you'll add on another tag, but do you want to try and create a 'fake tag' and then delete it afterwards? - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGwnoO4QvfyHIvDvMRArHSAJ4qYuexUGn6Ff3/PkHU/DjJBlCQnwCgxyy9 Qc852wMVRRMYdwoD3+p7X3Y= =FCD9 -END PGP SIGNATURE- ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] CVS corruption/mistagging?
Marc G. Fournier [EMAIL PROTECTED] writes: That was a good thing, but for security's sake these files ought to be chown'd to some existing committer's account. I can do a quick chown -R scrappy on the whole repository ... ok? Seems close enough, but please keep that tarball around for awhile just in case we need to look at the prior state. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] CVS corruption/mistagging?
Marc G. Fournier [EMAIL PROTECTED] writes: It should be done ... if you try and create a tag, it should generate an error message ... Uh, nope: $ cvs tag fooey README T README $ cvs log README | more RCS file: /cvsroot/pgsql/src/backend/optimizer/README,v Working file: README head: 1.39 branch: locks: strict access list: symbolic names: fooey: 1.39 REL7_3_19: 1.25 REL7_4_17: 1.29 REL8_0_13: 1.30 REL8_1_9: 1.33 REL8_2_4: 1.35.2.1 REL8_0_12: 1.30 REL8_1_8: 1.33 REL8_2_3: 1.35 REL7_3_18: 1.25 REL7_4_16: 1.29 REL8_0_11: 1.30 REL8_1_7: 1.33 REL8_2_2: 1.35 REL8_0_10: 1.30 cvs [log aborted]: received broken pipe signal $ cvs tag -d fooey README D README $ regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] CVS corruption/mistagging?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 'k, let me play with it ... - --On Wednesday, August 15, 2007 00:13:11 -0400 Tom Lane [EMAIL PROTECTED] wrote: Marc G. Fournier [EMAIL PROTECTED] writes: It should be done ... if you try and create a tag, it should generate an error message ... Uh, nope: $ cvs tag fooey README T README $ cvs log README | more RCS file: /cvsroot/pgsql/src/backend/optimizer/README,v Working file: README head: 1.39 branch: locks: strict access list: symbolic names: fooey: 1.39 REL7_3_19: 1.25 REL7_4_17: 1.29 REL8_0_13: 1.30 REL8_1_9: 1.33 REL8_2_4: 1.35.2.1 REL8_0_12: 1.30 REL8_1_8: 1.33 REL8_2_3: 1.35 REL7_3_18: 1.25 REL7_4_16: 1.29 REL8_0_11: 1.30 REL8_1_7: 1.33 REL8_2_2: 1.35 REL8_0_10: 1.30 cvs [log aborted]: received broken pipe signal $ cvs tag -d fooey README D README $ regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGwn9q4QvfyHIvDvMRAuqOAJwPcIcsfQ74wBwdjBR478C72nr+bwCfd8PY EeH2sh+biTfoFbTF2KxqYVU= =iMwZ -END PGP SIGNATURE- ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] CVS corruption/mistagging?
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
One better (and done now):
find . -nouser -exec chown scrappy {} \;
so, all files that had valid users owning them stay with those ...
- --On Wednesday, August 15, 2007 00:08:07 -0400 Tom Lane [EMAIL PROTECTED]
wrote:
Marc G. Fournier [EMAIL PROTECTED] writes:
That was a good thing, but for security's sake these files ought to be
chown'd to some existing committer's account.
I can do a quick chown -R scrappy on the whole repository ... ok?
Seems close enough, but please keep that tarball around for awhile just
in case we need to look at the prior state.
regards, tom lane
-
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED]
Yahoo . yscrappy Skype: hub.orgICQ . 7615664
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.4 (FreeBSD)
iD8DBQFGwonj4QvfyHIvDvMRAhNBAJ9b6Y1o4eICLrtuWncRIqN8g07XggCgnnc8
ttJoOScGbFEY/y3qP3MZYWY=
=RUO5
-END PGP SIGNATURE-
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [HACKERS] CVS corruption/mistagging?
Marc G. Fournier [EMAIL PROTECTED] writes:
One better (and done now):
find . -nouser -exec chown scrappy {} \;
[ eyeballs repository ... ] Check, that looks great from here.
regards, tom lane
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
