Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Sun, Sep 19, 2010 at 18:52, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 09/19/2010 12:25 PM, Tom Lane wrote: # We don't want to change line numbers, so we simply reduce the keyword # string to the file pathname part. For example, # $PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30 momjian Exp $ # becomes # $PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30 momjian Exp $ These before and after lines look identical to me. Sigh ... obviously didn't finish editing the comment :-( Of course the last line should read # src/port/unsetenv.c I've applied those to my repo, and am now re-running a final conversion before we do the live one. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
I wrote:
I looked a bit more at your pggit_migrate stuff. I'm not terribly happy
with the proposed clean_keywords.pl script. I'd like it to reduce the
$PostgreSQL$ thingies to the full pathname of the file, rather than
try to remove all trace of them, eg
* $PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30
momjian Exp $
becomes
* src/port/unsetenv.c
This would then be followed up by moving those pathname comments to
somewhere more sensible. I don't think that part can be managed with
a script like this, but leaving the data in place will make it easier
to do the moving. Some places, like the .sgml files, won't need any
additional changing to get to where I would like to be.
Also, I'd be inclined to make these changes only in master, not in the
back branches. We don't for example run pg_indent against back branches.
Pursuant to that, attached are proposed modified versions of the two
scripts involved.
regards, tom lane
#!/usr/bin/perl -w
#
# Attempt to remove all cvs keywords in the given directory tree
# (with all keywords meaning $PostgreSQL$ keyword)
#
# We don't want to change line numbers, so we simply reduce the keyword
# string to the file pathname part. For example,
# $PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30 momjian Exp $
# becomes
# $PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30 momjian Exp $
#
$REPODIR=$ARGV[0] || die No repository specified\n;
chdir($REPODIR) || die Could not chdir to $REPODIR\n;
open(L,git grep -l \\\$PostgreSQL |) || die Could not git-grep\n;
while (L) {
chomp;
my $fn = $_;
my $txt;
open(F,$fn) || die Could not read $_\n;
while (F) {
s|\$PostgreSQL: pgsql/(\S+),v [^\$]+\$|$1|;
$txt .= $_;
}
close(F);
open(F,$fn) || die Could not write $_\n;
print F $txt;
close(F);
$txt = '';
}
#!/bin/bash
set -e
REPO=/opt/gitrepo_cvs2git
HERE=$(pwd)
# clean master only
BRANCHES=master
cd $REPO
for B in $BRANCHES ; do
if [ $B != master ]; then
echo Creating branch $B
git branch -f $B --track origin/$B
fi
echo Switching to $B
git checkout $B
echo Cleaning $B
perl $HERE/clean_keywords.pl $REPO
echo Committing cleanup
git commit -a -F - EOF
Remove cvs keywords from all files.
EOF
done
echo All branches updated, don't forget to push!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On 09/19/2010 12:25 PM, Tom Lane wrote: Pursuant to that, attached are proposed modified versions of the two scripts involved. # # We don't want to change line numbers, so we simply reduce the keyword # string to the file pathname part. For example, # $PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30 momjian Exp $ # becomes # $PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30 momjian Exp $ # These before and after lines look identical to me. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Andrew Dunstan and...@dunslane.net writes: On 09/19/2010 12:25 PM, Tom Lane wrote: # We don't want to change line numbers, so we simply reduce the keyword # string to the file pathname part. For example, # $PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30 momjian Exp $ # becomes # $PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30 momjian Exp $ These before and after lines look identical to me. Sigh ... obviously didn't finish editing the comment :-( Of course the last line should read # src/port/unsetenv.c regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
I wrote: This commit occurred during that interval between where we'd initially branched 7.3 and where we moved it up to head. I think what happened was that I tried to back-patch a fix into what was then the 7.3 branch, and when Marc moved the branch point, these commits all ended up orphans. So indeed there's no harm discarding them. (I always suspected that what Marc had done wasn't entirely kosher, and yup, now the chickens are coming home to roost.) BTW, if you're wondering why there was only one such commit, it's because we decided to not maintain the branch only a day after Marc had sprouted it. See discussion here: http://archives.postgresql.org/pgsql-hackers/2002-09/msg01733.php regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Tue, Sep 14, 2010 at 12:01:18PM -0400, Tom Lane wrote: Well ... I guess the other attitude we could take is that that was a private development branch of Michael's. If we'd been working in git Actually it wasn't. This branch was created when ecpg grew too big for the released version of bison. We had to use a development snapshot of bison to even compile ecpg back then and didn't want to put the ecpg changes into the main branch because this would have prevented most people from compiling PG. At least that's what I still remember. As far as converting this branch to git, I'd say forget about it. All changes to the parser went into CVS HEAD as soon as a suitable bison version was released. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber: michael.meskes at googlemail dot com VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Tue, Sep 14, 2010 at 10:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: Attached is an updated repository.fixups script that inserts dead revisions in every case where a new file was back-patched into an existing branch. With that, we are down to a total of nine manufactured commits, to wit: [details] Magnus posted an updated conversion this morning. http://git.postgresql.org/gitweb?p=postgresql-migration.git;a=summary Evidently, however, he didn't do the same things you did, because there are DEFINITELY more than 9 manufactured commits in this one. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Robert Haas robertmh...@gmail.com writes: Magnus posted an updated conversion this morning. http://git.postgresql.org/gitweb?p=postgresql-migration.git;a=summary Evidently, however, he didn't do the same things you did, because there are DEFINITELY more than 9 manufactured commits in this one. Um ... I just did git clone git://git.postgresql.org/git/postgresql-migration.git and I only see nine. It's got some *other* problems though; compared to a conversion I just finished locally, it's missing a whole lot of history for some of the old jdbc files. Is there any possibility that git clone isn't very trustworthy? It's a bit scary that we don't see identical views of this repository. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Fri, Sep 17, 2010 at 11:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Magnus posted an updated conversion this morning. http://git.postgresql.org/gitweb?p=postgresql-migration.git;a=summary Evidently, however, he didn't do the same things you did, because there are DEFINITELY more than 9 manufactured commits in this one. Um ... I just did git clone git://git.postgresql.org/git/postgresql-migration.git and I only see nine. It's got some *other* problems though; compared to a conversion I just finished locally, it's missing a whole lot of history for some of the old jdbc files. Is there any possibility that git clone isn't very trustworthy? It's a bit scary that we don't see identical views of this repository. *scratches head* I did a git-fetch into an existing copy of the old contents of that repository, rather than a fresh clone. Let me nuke it and start over. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On 09/17/2010 11:39 AM, Tom Lane wrote: Is there any possibility that git clone isn't very trustworthy? It's a bit scary that we don't see identical views of this repository. I should have thought that very unlikely. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Fri, Sep 17, 2010 at 11:55 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Sep 17, 2010 at 11:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Magnus posted an updated conversion this morning. http://git.postgresql.org/gitweb?p=postgresql-migration.git;a=summary Evidently, however, he didn't do the same things you did, because there are DEFINITELY more than 9 manufactured commits in this one. Um ... I just did git clone git://git.postgresql.org/git/postgresql-migration.git and I only see nine. It's got some *other* problems though; compared to a conversion I just finished locally, it's missing a whole lot of history for some of the old jdbc files. Is there any possibility that git clone isn't very trustworthy? It's a bit scary that we don't see identical views of this repository. *scratches head* I did a git-fetch into an existing copy of the old contents of that repository, rather than a fresh clone. Let me nuke it and start over. OK, the fresh clone does in fact show just 9 manufactured commits. Sorry, I must not have cleaned out the old state properly. What's the problem with the old JDBC files? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Robert Haas robertmh...@gmail.com writes: On Fri, Sep 17, 2010 at 11:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: and I only see nine. It's got some *other* problems though; compared to a conversion I just finished locally, it's missing a whole lot of history for some of the old jdbc files. What's the problem with the old JDBC files? This is what I sent to Magnus off-list: There seems to be something wrong with this :-(. I pulled this down and compared the output of git log --all --source --name-status to what I got from a fresh conversion of my own. There seems to be a substantial loss of history around some of the jdbc files. See attached diffs from my log to yours. regards, tom lane *** revhist.git17 Fri Sep 17 10:44:50 2010 --- revhist.gitmha Fri Sep 17 11:24:03 2010 *** *** 236673,236679 From Donald Fraser. - M src/interfaces/jdbc/org/postgresql/core/QueryExecutor.java M src/interfaces/jdbc/org/postgresql/fastpath/Fastpath.java M src/interfaces/jdbc/org/postgresql/jdbc1/AbstractJdbc1Connection.java --- 236673,236678 *** *** 249776,249786 jdbc/org/postgresql/test/jdbc2/ServerCursorTest.java M src/interfaces/jdbc/org/postgresql/Driver.java.in - M src/interfaces/jdbc/org/postgresql/core/BaseResultSet.java - M src/interfaces/jdbc/org/postgresql/core/BaseStatement.java - M src/interfaces/jdbc/org/postgresql/core/Field.java - M src/interfaces/jdbc/org/postgresql/core/PGStream.java - M src/interfaces/jdbc/org/postgresql/core/QueryExecutor.java M src/interfaces/jdbc/org/postgresql/jdbc1/AbstractJdbc1DatabaseMetaData.java M src/interfaces/jdbc/org/postgresql/jdbc1/AbstractJdbc1ResultSet.java M src/interfaces/jdbc/org/postgresql/jdbc1/AbstractJdbc1Statement.java --- 249775,249780 *** *** 254896,254902 Per report from Hans Nather. - M src/interfaces/jdbc/org/postgresql/core/QueryExecutor.java M src/interfaces/jdbc/org/postgresql/fastpath/Fastpath.java M src/interfaces/jdbc/org/postgresql/test/jdbc2/Jdbc2TestSuite.java A src/interfaces/jdbc/org/postgresql/test/jdbc2/NotifyTest.java --- 254890,254895 *** *** 256307,256321 D src/interfaces/jdbc/org/postgresql/PGNotification.java D src/interfaces/jdbc/org/postgresql/PGRefCursorResultSet.java D src/interfaces/jdbc/org/postgresql/PGStatement.java - D src/interfaces/jdbc/org/postgresql/core/BaseConnection.java - D src/interfaces/jdbc/org/postgresql/core/BaseResultSet.java - D src/interfaces/jdbc/org/postgresql/core/BaseStatement.java - D src/interfaces/jdbc/org/postgresql/core/Encoding.java - D src/interfaces/jdbc/org/postgresql/core/Field.java - D src/interfaces/jdbc/org/postgresql/core/Notification.java - D src/interfaces/jdbc/org/postgresql/core/PGStream.java - D src/interfaces/jdbc/org/postgresql/core/QueryExecutor.java - D src/interfaces/jdbc/org/postgresql/core/StartupPacket.java D src/interfaces/jdbc/org/postgresql/errors.properties D src/interfaces/jdbc/org/postgresql/errors_de.properties D src/interfaces/jdbc/org/postgresql/errors_fr.properties --- 256300,256305 *** *** 261723,261729 M src/interfaces/jdbc/example/corba/StockServer.java M src/interfaces/jdbc/example/corba/stock.idl M src/interfaces/jdbc/example/corba/stock.sql - M src/interfaces/jdbc/org/postgresql/core/StartupPacket.java M src/interfaces/jdbc/org/postgresql/test/jdbc2/BlobTest.java M src/interfaces/jdbc/org/postgresql/test/jdbc2/ConnectionTest.java M src/interfaces/jdbc/org/postgresql/test/jdbc2/DatabaseMetaDataTest.java --- 261707,261712 *** *** 262635,262648 M src/interfaces/jdbc/org/postgresql/PGNotification.java M src/interfaces/jdbc/org/postgresql/PGRefCursorResultSet.java M src/interfaces/jdbc/org/postgresql/PGStatement.java - M src/interfaces/jdbc/org/postgresql/core/BaseConnection.java - M src/interfaces/jdbc/org/postgresql/core/BaseResultSet.java - M src/interfaces/jdbc/org/postgresql/core/BaseStatement.java - M src/interfaces/jdbc/org/postgresql/core/Encoding.java - M src/interfaces/jdbc/org/postgresql/core/Field.java - M src/interfaces/jdbc/org/postgresql/core/Notification.java - M src/interfaces/jdbc/org/postgresql/core/PGStream.java - M src/interfaces/jdbc/org/postgresql/core/QueryExecutor.java M src/interfaces/jdbc/org/postgresql/errors_de.properties M src/interfaces/jdbc/org/postgresql/fastpath/Fastpath.java M src/interfaces/jdbc/org/postgresql/fastpath/FastpathArg.java --- 262618,262623 *** *** 266110,266117 Patches from Oliver Jowett to fix CursorFetchTest, 7.4 now does not automatically delete cursors - M
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Fri, Sep 17, 2010 at 18:28, Robert Haas robertmh...@gmail.com wrote: On Fri, Sep 17, 2010 at 11:55 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Sep 17, 2010 at 11:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Magnus posted an updated conversion this morning. http://git.postgresql.org/gitweb?p=postgresql-migration.git;a=summary Evidently, however, he didn't do the same things you did, because there are DEFINITELY more than 9 manufactured commits in this one. Um ... I just did git clone git://git.postgresql.org/git/postgresql-migration.git and I only see nine. It's got some *other* problems though; compared to a conversion I just finished locally, it's missing a whole lot of history for some of the old jdbc files. Is there any possibility that git clone isn't very trustworthy? It's a bit scary that we don't see identical views of this repository. *scratches head* I did a git-fetch into an existing copy of the old contents of that repository, rather than a fresh clone. Let me nuke it and start over. OK, the fresh clone does in fact show just 9 manufactured commits. Sorry, I must not have cleaned out the old state properly. Turns out I did th esame thing from my box to the repo on git.postgresql.org. So I've now wiped that repository and re-pushed mine. Can you give it another check? (from a fresh clone) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Magnus Hagander mag...@hagander.net writes: On Fri, Sep 17, 2010 at 18:28, Robert Haas robertmh...@gmail.com wrote: Sorry, I must not have cleaned out the old state properly. Turns out I did th esame thing from my box to the repo on git.postgresql.org. So I've now wiped that repository and re-pushed mine. Can you give it another check? (from a fresh clone) I re-cloned but got the exact same state as before --- jdbc still wonky. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Fri, Sep 17, 2010 at 19:20, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Fri, Sep 17, 2010 at 18:28, Robert Haas robertmh...@gmail.com wrote: Sorry, I must not have cleaned out the old state properly. Turns out I did th esame thing from my box to the repo on git.postgresql.org. So I've now wiped that repository and re-pushed mine. Can you give it another check? (from a fresh clone) I re-cloned but got the exact same state as before --- jdbc still wonky. That's weird. Just to confirm, you ran your patch against current cvs, right? So you also got the hunk succeeded at offset 1 line a whole bunch of times? Then it's not that that's broken. And I'm on cvs2git revision 5270. The script I've run is on http://github.com/mhagander/pggit_migrate - it's the migrate_cvs.sh script. The repository_fixups script is a direct import of yours except I added a set -e at the start. (the version pushed hasn't had the git gc step run, but that's the only one that differs) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Magnus Hagander mag...@hagander.net writes: Just to confirm, you ran your patch against current cvs, right? So you also got the hunk succeeded at offset 1 line a whole bunch of times? Then it's not that that's broken. Right, the patch still applies fine, it's just off by a line or so in many places (probably because of the new REL9_0_0 tags). The script I've run is on http://github.com/mhagander/pggit_migrate - it's the migrate_cvs.sh script. The repository_fixups script is a direct import of yours except I added a set -e at the start. Hmm. I didn't try rsync'ing from anoncvs ... I logged into the master and tar'd up the /cvsroot directory ;-). I wonder if there's something wrong with the anoncvs copy of that subdirectory? Will do the rsync and compare. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
I wrote: Hmm. I didn't try rsync'ing from anoncvs ... I logged into the master and tar'd up the /cvsroot directory ;-). I wonder if there's something wrong with the anoncvs copy of that subdirectory? Will do the rsync and compare. Doh: Only in myrepo/pgsql/contrib/retep/uk/org/retep/xml: core Only in myrepo/pgsql/src/interfaces/jdbc/org/postgresql: core Something in the rsync process thinks that ignoring subdirectories named core is a good idea. I'm a bit surprised nobody ever noticed these were missing from anoncvs before ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Fri, Sep 17, 2010 at 20:24, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: Hmm. I didn't try rsync'ing from anoncvs ... I logged into the master and tar'd up the /cvsroot directory ;-). I wonder if there's something wrong with the anoncvs copy of that subdirectory? Will do the rsync and compare. Doh: Only in myrepo/pgsql/contrib/retep/uk/org/retep/xml: core Only in myrepo/pgsql/src/interfaces/jdbc/org/postgresql: core Something in the rsync process thinks that ignoring subdirectories named core is a good idea. I'm a bit surprised nobody ever noticed these were missing from anoncvs before ... Oh FFS.. It's the adding of -C to the commandline to rsync. And it's not added when syncing to anoncvs - it's there on anoncvs. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
I wrote: Something in the rsync process thinks that ignoring subdirectories named core is a good idea. I'm a bit surprised nobody ever noticed these were missing from anoncvs before ... That's because they aren't. It's the -C switch in your rsync call that's at fault. (And this demonstrates why comparing against checkouts from the same CVS repo isn't an end-to-end test :-() regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Fri, Sep 17, 2010 at 20:32, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: Something in the rsync process thinks that ignoring subdirectories named core is a good idea. I'm a bit surprised nobody ever noticed these were missing from anoncvs before ... That's because they aren't. It's the -C switch in your rsync call that's at fault. (And this demonstrates why comparing against checkouts from the same CVS repo isn't an end-to-end test :-() Yeah, I noticed that too. Re-running now. FWIW, I got thecommand off the wiki's recommendations for how to use rsync -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
I looked a bit more at your pggit_migrate stuff. I'm not terribly happy with the proposed clean_keywords.pl script. I'd like it to reduce the $PostgreSQL$ thingies to the full pathname of the file, rather than try to remove all trace of them, eg *$PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30 momjian Exp $ becomes *src/port/unsetenv.c This would then be followed up by moving those pathname comments to somewhere more sensible. I don't think that part can be managed with a script like this, but leaving the data in place will make it easier to do the moving. Some places, like the .sgml files, won't need any additional changing to get to where I would like to be. Also, I'd be inclined to make these changes only in master, not in the back branches. We don't for example run pg_indent against back branches. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On 09/17/2010 02:37 PM, Magnus Hagander wrote: On Fri, Sep 17, 2010 at 20:32, Tom Lanet...@sss.pgh.pa.us wrote: I wrote: Something in the rsync process thinks that ignoring subdirectories named core is a good idea. I'm a bit surprised nobody ever noticed these were missing from anoncvs before ... That's because they aren't. It's the -C switch in your rsync call that's at fault. (And this demonstrates why comparing against checkouts from the same CVS repo isn't an end-to-end test :-() Yeah, I noticed that too. Re-running now. FWIW, I got thecommand off the wiki's recommendations for how to use rsync This is what I have been using for a very long time: rsync -avzH --delete --exclude-from=/home/cvsmirror/pg-exclude anoncvs.postgresql.org::pgsql-cvs /home/cvsmirror/pg The exclude file contains: /sup/ /CVSROOT/loginfo* /CVSROOT/commitinfo* /CVSROOT/config* /CVSROOT/passwd /CVSROOT/history cheers andrew
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Fri, Sep 17, 2010 at 20:49, Tom Lane t...@sss.pgh.pa.us wrote: I looked a bit more at your pggit_migrate stuff. I'm not terribly happy with the proposed clean_keywords.pl script. I'd like it to reduce the $PostgreSQL$ thingies to the full pathname of the file, rather than try to remove all trace of them, eg * $PostgreSQL: pgsql/src/port/unsetenv.c,v 1.12 2010/09/07 14:10:30 momjian Exp $ becomes * src/port/unsetenv.c This would then be followed up by moving those pathname comments to somewhere more sensible. I don't think that part can be managed with a script like this, but leaving the data in place will make it easier to do the moving. Some places, like the .sgml files, won't need any additional changing to get to where I would like to be. Also, I'd be inclined to make these changes only in master, not in the back branches. We don't for example run pg_indent against back branches. We discussed that before, and the consensus then was to do that, and backport it to *active* backbranches. Just to not demove the lines, but replace them with an empty one so that line numbers would stay the same. But it's not too late to change that again... Shouldn't be too hard to change the script to change it the way you suggest, and it's *certainly* not hard to restrict it to just to the master branch... Will not doing the backbranches make it harder to backport patches? Probably shouldn't, unless you're changing the very first line of the file, right? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Magnus Hagander mag...@hagander.net writes: Will not doing the backbranches make it harder to backport patches? Probably shouldn't, unless you're changing the very first line of the file, right? The $PostgreSQL$ lines haven't been a backporting problem in the past, so I don't see why they'd be one now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Fri, Sep 17, 2010 at 20:37, Magnus Hagander mag...@hagander.net wrote: On Fri, Sep 17, 2010 at 20:32, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: Something in the rsync process thinks that ignoring subdirectories named core is a good idea. I'm a bit surprised nobody ever noticed these were missing from anoncvs before ... That's because they aren't. It's the -C switch in your rsync call that's at fault. (And this demonstrates why comparing against checkouts from the same CVS repo isn't an end-to-end test :-() Yeah, I noticed that too. Re-running now. Ok, re-run off a correct rsync pushed. How does it look now? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
BTW, on the cleanup steps: # Remove bogus branches git branch -D unlabeled-1.44.2 git branch -D unlabeled-1.51.2 git branch -D unlabeled-1.59.2 git branch -D unlabeled-1.87.2 git branch -D unlabeled-1.90.2 You should not need any of the above; I don't see those being generated anymore with the cleanup script in place. git branch -D ecpg_big_bison I don't agree with removing this. It is a legitimate part of the project history. Yeah, it was a workaround, but we've had workarounds in place for other broken software. # Remove broken tags git tag -d SUPPORT git tag -d MANUAL_1_0 git tag -d Release-1-6-0 Also get rid of the creation tag, please. Also, just for the record, these aren't really broken. Maybe Remove unwanted partial tags? In addition to the above, we're going to want to clean up the Release_2_0_0 and Release_2_0 tags, but I'm not sure if there's a reasonable way to script those when the commit SHA1's aren't frozen yet. I can give you timestamps for the commits they should point at, but I lack the git-fu to convert that into a git tag command. Similarly, we're going to want to repoint REL6_5, REL7_1, REL7_1_2 tags at more appropriate places, but right now all I have for those is timestamps. (I've also identified places to tag the other early releases, but those tags can certainly be added later.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
I wrote: In addition to the above, we're going to want to clean up the Release_2_0_0 and Release_2_0 tags, but I'm not sure if there's a reasonable way to script those when the commit SHA1's aren't frozen yet. I can give you timestamps for the commits they should point at, but I lack the git-fu to convert that into a git tag command. Similarly, we're going to want to repoint REL6_5, REL7_1, REL7_1_2 tags at more appropriate places, but right now all I have for those is timestamps. (I've also identified places to tag the other early releases, but those tags can certainly be added later.) Actually, the simplest way to handle this might be to just delete all five of those tags during the conversion, and then I'll put them back in the right places later when I add the other old-release tags. That way we won't have any tags getting moved after the repository is published. (Or am I wrong about that being something to avoid? But in any case we want to gc the manufactured commits for Release_2_0_0 and Release_2_0.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Fri, Sep 17, 2010 at 23:01, Tom Lane t...@sss.pgh.pa.us wrote: BTW, on the cleanup steps: # Remove bogus branches git branch -D unlabeled-1.44.2 git branch -D unlabeled-1.51.2 git branch -D unlabeled-1.59.2 git branch -D unlabeled-1.87.2 git branch -D unlabeled-1.90.2 You should not need any of the above; I don't see those being generated anymore with the cleanup script in place. I see them - at least the script didn't give an error when it ran, but said it had deleted them. git branch -D ecpg_big_bison I don't agree with removing this. It is a legitimate part of the project history. Yeah, it was a workaround, but we've had workarounds in place for other broken software. Ok. I was just going by Michaels email - I'll leave it in then. # Remove broken tags git tag -d SUPPORT git tag -d MANUAL_1_0 git tag -d Release-1-6-0 Also get rid of the creation tag, please. Also, just for the record, these aren't really broken. Maybe Remove unwanted partial tags? Right. I didn't really pay much attention to the comments, since it's just a temporary thing. creation added to the list. In addition to the above, we're going to want to clean up the Release_2_0_0 and Release_2_0 tags, but I'm not sure if there's a reasonable way to script those when the commit SHA1's aren't frozen yet. I can give you timestamps for the commits they should point at, but I lack the git-fu to convert that into a git tag command. yeah, that's definitely easier to do after the fact. Similarly, we're going to want to repoint REL6_5, REL7_1, REL7_1_2 tags at more appropriate places, but right now all I have for those is timestamps. (I've also identified places to tag the other early releases, but those tags can certainly be added later.) I like your downthread suggestion of removing them, and then putting them back in manually later. I'll add them all to the delete list. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Fri, Sep 17, 2010 at 23:21, Tom Lane t...@sss.pgh.pa.us wrote: Actually, the simplest way to handle this might be to just delete all five of those tags during the conversion, and then I'll put them back in the right places later when I add the other old-release tags. That way we won't have any tags getting moved after the repository is published. (Or am I wrong about that being something to avoid? But in any case we want to gc the manufactured commits for Release_2_0_0 and Release_2_0.) As long as nobody has cloned the repository, it's not a problem moving them. But it *is* something you should generally avoid, so let's do that :-) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Magnus Hagander mag...@hagander.net writes: On Fri, Sep 17, 2010 at 23:01, Tom Lane t...@sss.pgh.pa.us wrote: git branch -D unlabeled-1.44.2 git branch -D unlabeled-1.51.2 git branch -D unlabeled-1.59.2 git branch -D unlabeled-1.87.2 git branch -D unlabeled-1.90.2 You should not need any of the above; I don't see those being generated anymore with the cleanup script in place. I see them - at least the script didn't give an error when it ran, but said it had deleted them. [ scratches head ... ] That's weird. We probably ought to figure out why you and I are getting different results. I wonder if there's some other discrepancy in the anoncvs pull? Anyway, if what you have up at http://github.com/mhagander/pggit_migrate is current, I can try to reproduce your results here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Sat, Sep 18, 2010 at 00:06, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Fri, Sep 17, 2010 at 23:01, Tom Lane t...@sss.pgh.pa.us wrote: git branch -D unlabeled-1.44.2 git branch -D unlabeled-1.51.2 git branch -D unlabeled-1.59.2 git branch -D unlabeled-1.87.2 git branch -D unlabeled-1.90.2 You should not need any of the above; I don't see those being generated anymore with the cleanup script in place. I see them - at least the script didn't give an error when it ran, but said it had deleted them. [ scratches head ... ] That's weird. We probably ought to figure out why you and I are getting different results. I wonder if there's some other discrepancy in the anoncvs pull? Could be. Anyway, if what you have up at http://github.com/mhagander/pggit_migrate is current, I can try to reproduce your results here. It is - I just double-checked that. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Magnus Hagander mag...@hagander.net writes: On Sat, Sep 18, 2010 at 00:06, Tom Lane t...@sss.pgh.pa.us wrote: [ scratches head ... ] That's weird. We probably ought to figure out why you and I are getting different results. I wonder if there's some other discrepancy in the anoncvs pull? Could be. Oh, mystery explained upon comparing the cvs2git.options files. I was using Max's file which had this in it: ExcludeRegexpStrategyRule(r'unlabeled-.*'), I think I'll rerun without that just to convince myself of what it is we're dropping. But right now it seems that everything is pretty sane. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
I wrote: Oh, mystery explained upon comparing the cvs2git.options files. I was using Max's file which had this in it: ExcludeRegexpStrategyRule(r'unlabeled-.*'), I think I'll rerun without that just to convince myself of what it is we're dropping. But right now it seems that everything is pretty sane. OK, I found out what we're dropping. Those unlabeled branches each have a manufactured creation commit that just deletes a lot of files, plus a single real commit, which all look like this: + commit 6c23cb5f371f273cade66408f3a888c2f94af293 refs/heads/unlabeled-1.51.2 + Author: Tom Lane t...@sss.pgh.pa.us + Date: Sat Sep 28 20:00:29 2002 + + + Make the world at least somewhat safe for zero-column tables, and + remove the special case in ALTER DROP COLUMN to prohibit dropping a + table's last column. + + M src/test/regress/expected/alter_table.out + + commit d554d56827e4337835b0792c2568e8e9fee2d950 refs/heads/unlabeled-1.87.2 + Author: Tom Lane t...@sss.pgh.pa.us + Date: Sat Sep 28 20:00:28 2002 + + + Make the world at least somewhat safe for zero-column tables, and + remove the special case in ALTER DROP COLUMN to prohibit dropping a + table's last column. + + M src/backend/executor/nodeAgg.c + + commit 7724eba3a121e5c84da3467fe6f0b176bf3f3d4b refs/heads/unlabeled-1.59.2 + Author: Tom Lane t...@sss.pgh.pa.us + Date: Sat Sep 28 20:00:27 2002 + + + Make the world at least somewhat safe for zero-column tables, and + remove the special case in ALTER DROP COLUMN to prohibit dropping a + table's last column. + + M src/backend/executor/execTuples.c + + commit 6924390f0cfa2aac1308719e51cb4cc2b24e2bc7 refs/heads/unlabeled-1.44.2 + Author: Tom Lane t...@sss.pgh.pa.us + Date: Sat Sep 28 20:00:26 2002 + + + Make the world at least somewhat safe for zero-column tables, and + remove the special case in ALTER DROP COLUMN to prohibit dropping a + table's last column. + + M src/backend/commands/tablecmds.c + + commit 83029b0525f3324c30d35fcc9c77d2c301bdf7cf refs/heads/unlabeled-1.90.2 + Author: Tom Lane t...@sss.pgh.pa.us + Date: Sat Sep 28 20:00:25 2002 + + + Make the world at least somewhat safe for zero-column tables, and + remove the special case in ALTER DROP COLUMN to prohibit dropping a + table's last column. + + M src/backend/access/common/tupdesc.c + M src/backend/parser/parse_target.c + commit 6d0d15c451739396851d3f93f81c63a47535bf1e refs/tags/REL7_4_BETA1 Author: Tom Lane t...@sss.pgh.pa.us Date: Sat Sep 28 20:00:19 2002 + If you go and look at the CVS history of these files, you'll see that indeed they all have branches sprouted on that date that are unreferenced. This commit occurred during that interval between where we'd initially branched 7.3 and where we moved it up to head. I think what happened was that I tried to back-patch a fix into what was then the 7.3 branch, and when Marc moved the branch point, these commits all ended up orphans. So indeed there's no harm discarding them. (I always suspected that what Marc had done wasn't entirely kosher, and yup, now the chickens are coming home to roost.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Tom Lane t...@sss.pgh.pa.us writes: PS: This attachment is text/x-patch instead of text/plain ... does it come through as an attachment for you, Robert? From my MUA, I can say that it's not so much a problem of MIME type than the Content-Disposition, yours are always inline. http://www.gnus.org/manual/emacs-mime_11.html#SEC11 http://en.wikipedia.org/wiki/MIME#Content-Disposition Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Tue, Sep 14, 2010 at 10:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: * Four that create the partial tags SUPPORT, MANUAL_1_0, creation, and Release-1-6-0. I think we agreed that we can just drop these tags and allow their manufactured commits to be garbage-collected. +1. * Two that create the tags Release_2_0 and Release_2_0_0. I think these probably represent a cvs2git bug, as there is no apparent reason why it didn't just apply the tags to the immediately preceding mainline commits instead. In any case, we can get rid of them by moving the tags to the appropriate commits manually. +1. * One that creates the branch REL2_0B. This is caused by a known, longstanding cvs2git deficiency: it fails to pick the optimal place to branch from when file deletions are involved. We're just going to have to live with that, I think; it's a pretty minor infelicity anyway. Fine with me. * One that creates the partial branch ecpg_big_bison. I think we have to live with this too. I don't want to drop the branch altogether, as that would represent a loss of development history. The only other alternative I can think of is to try to convert it into a full branch, but I'm unsure what the implications would be of that. I doubt there's a clean way to do that. I am not sure there's much point in moving the tag over to git - anyone wanting to do something useful with it will need to use CVS anyway, won't they? * And lastly, there's a weird manufactured commit that adds a passel of files on REL7_3_STABLE branch, only to have them deleted again by the following real commit. This is a result of the fact that the branch point was moved long after creation, as discussed here: http://archives.postgresql.org/pgsql-hackers/2002-11/msg00127.php We could maybe try to get rid of both the manufactured commit and the deletion commit, but I'm inclined not to. The underlying history is really as dirty as this commit makes it look. OK. The long and the short of it is that I'm now satisfied with the git conversion. There is still the issue of adding/adjusting release tags for ancient releases, but the lack of those is surely not the conversion's fault. Great. PS: This attachment is text/x-patch instead of text/plain ... does it come through as an attachment for you, Robert? Yep, thanks. I'd like to have Magnus run a test conversion with all the latest and greatest stuff and throw it up somewhere so we can all poke at it. Incidentally, with respect to timing, do we want to press on with this conversion now or wait until after the CommitFest is done? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Robert Haas robertmh...@gmail.com writes: Incidentally, with respect to timing, do we want to press on with this conversion now or wait until after the CommitFest is done? I'd kind of like to do it before we start the commitfest. These repository patches will go stale if we wait too long, and a month is probably too long. In any case I'd rather get it done while all the information is fresh in mind. The main schedule constraint I can see at the moment is that 9.0 wrap is scheduled for Thursday, and I think we probably don't want to do it before the wrap. Another issue is that we need a chunk of Magnus' time to shepherd the conversion, and I don't know what his availability is. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Robert Haas robertmh...@gmail.com writes: On Tue, Sep 14, 2010 at 10:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: * One that creates the partial branch ecpg_big_bison. I think we have to live with this too. I don't want to drop the branch altogether, as that would represent a loss of development history. The only other alternative I can think of is to try to convert it into a full branch, but I'm unsure what the implications would be of that. I doubt there's a clean way to do that. I am not sure there's much point in moving the tag over to git - anyone wanting to do something useful with it will need to use CVS anyway, won't they? Well ... I guess the other attitude we could take is that that was a private development branch of Michael's. If we'd been working in git at the time, that branch would never have been seen outside his personal repository, most likely. The changes did eventually get merged back to HEAD, so we'd not be losing anything critical if we just dropped the branch altogether. Anybody else have an opinion on what to do with it? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Excerpts from Dimitri Fontaine's message of mar sep 14 11:10:50 -0400 2010: Tom Lane t...@sss.pgh.pa.us writes: PS: This attachment is text/x-patch instead of text/plain ... does it come through as an attachment for you, Robert? From my MUA, I can say that it's not so much a problem of MIME type than the Content-Disposition, yours are always inline. Hmm, I see it as a separate attachment in this case. The original mail was indeed collapsed in that all the text attachments looked like a single text stream. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Tue, Sep 14, 2010 at 12:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Sep 14, 2010 at 10:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: * One that creates the partial branch ecpg_big_bison. I think we have to live with this too. I don't want to drop the branch altogether, as that would represent a loss of development history. The only other alternative I can think of is to try to convert it into a full branch, but I'm unsure what the implications would be of that. I doubt there's a clean way to do that. I am not sure there's much point in moving the tag over to git - anyone wanting to do something useful with it will need to use CVS anyway, won't they? Well ... I guess the other attitude we could take is that that was a private development branch of Michael's. If we'd been working in git at the time, that branch would never have been seen outside his personal repository, most likely. The changes did eventually get merged back to HEAD, so we'd not be losing anything critical if we just dropped the branch altogether. Anybody else have an opinion on what to do with it? We're not planning to delete the CVS repository, are we? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Robert Haas robertmh...@gmail.com writes: We're not planning to delete the CVS repository, are we? Not in the short term, but I'd like to think that the git repository will contain everything of conceivable interest. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Tue, Sep 14, 2010 at 12:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: We're not planning to delete the CVS repository, are we? Not in the short term, but I'd like to think that the git repository will contain everything of conceivable interest. Hmm, OK. That's never really been one of my goals. :-) I want a good, clean, complete history in git, but ancient partial branches are below my threshold for caring. But if you feel it's useful, we can keep the tag - I don't care enough to argue about it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Robert Haas robertmh...@gmail.com writes: On Tue, Sep 14, 2010 at 12:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: We're not planning to delete the CVS repository, are we? Not in the short term, but I'd like to think that the git repository will contain everything of conceivable interest. Hmm, OK. That's never really been one of my goals. :-) I will confess to moving the goal posts a bit ;-). If we didn't have such a near-perfect conversion, I would be willing to throw stuff overboard on the grounds that people could go back to the CVS repository if they cared. But we are at a point now where it's very hard to conceive of a reason for needing to do that. So I don't want to arbitrarily create reasons. I want a good, clean, complete history in git, but ancient partial branches are below my threshold for caring. But if you feel it's useful, we can keep the tag - I don't care enough to argue about it. ... but having said that, I'm not sure that the ecpg_big_bison branch should be considered part of the core project history. You could certainly argue that it wouldn't be there anyway if we'd had better tools. Again, I'd be interested to hear some other people's opinions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Excerpts from Tom Lane's message of mar sep 14 12:45:28 -0400 2010: I want a good, clean, complete history in git, but ancient partial branches are below my threshold for caring. But if you feel it's useful, we can keep the tag - I don't care enough to argue about it. ... but having said that, I'm not sure that the ecpg_big_bison branch should be considered part of the core project history. You could certainly argue that it wouldn't be there anyway if we'd had better tools. Again, I'd be interested to hear some other people's opinions. I think there's hardly much of interest in that branch, so it doesn't make sense to waste too much effort on it. However, why would we delete it? Just keep it with the manufactured tag and all -- so it is there, even if the history is not all that clean. This _is_ an option, right? -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Alvaro Herrera alvhe...@commandprompt.com writes: I think there's hardly much of interest in that branch, so it doesn't make sense to waste too much effort on it. However, why would we delete it? Just keep it with the manufactured tag and all -- so it is there, even if the history is not all that clean. This _is_ an option, right? That was pretty much my default position; I was just wondering if there was a consensus to do something else. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Sun, Sep 12, 2010 at 11:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: I've spent much of the weekend examining the discrepancies between our CVS repository and the tarballs available from our FTP archives, and after that trying to remove infelicities in the cvs2git output. There are a couple of remaining oddities that I would classify as probable cvs2git bugs, but an awful lot of it is inconsistencies in the CVS repository itself, some of which I can explain and some that I can't. Read on for many boring details. First of all, WOW, and thank you very much for putting in the time to make this happen. With those changes, I am able to match all the available archival tarballs to various places in the CVS history. The exact spots where they match are detailed in the attached matches file. The file also shows the Regrettably, all of your attachments came through as part of the actual email, both in my GMail and in the archives. I hate technology. Having completed that comparison, I then moved on to trying to get rid of the discrepancies in the git conversion; particularly, trying to get rid of the manufactured commits. I didn't have much success in that for the cases where the manufactured commit was caused by a back-branch file addition. [...] We still have manufactured commits either way, but they are just cosmetic so I guess we should live with them. I'm not really following what the history looks like here. What are the contents (git show) of the manufactured commit? I also found numerous places where we'd been sloppy about placing tags. That explains some of the weird things cvs2git did. In particular: * We had the already-known problem that gram.c and some other derived files had commits made after they should have been dead. * Bruce had transiently added those files on the WIN32_DEV branch as well, to general disapproval, and this seemed to also give cvs2git indigestion. The attached proposed fixup script deals with this by deleting those revisions altogether. This is a loss of history, but not one that I care about. * The HISTORY and INSTALL files have REL7_3_10 tags and should not. As mentioned earlier, I think this is because they were deleted after the original placement of that tag, and weren't correctly fixed when the tag was moved up to branch end a few days later. * The regression tests files recently added to contrib/xml2 have REL8_0_23 tags. I have no idea how that happened, because they certainly didn't exist when 8.0.23 was released. * There are a bunch of files that should have REL7_3_5 tags and lack them. They are in just a few subdirectories, so probably what happened was that the cvs tag operation was issued in an incomplete checkout tree. * Similarly, gram.c should have a release-6-3 tag and lacks it. * There are a bunch of files that have REL7_1 tags when what they should have are REL7_1_BETA tags. These appear to be exactly the files that were deleted between the initial placement of the REL7_1 tag and Marc's later ex-post-facto renaming of the tag to REL7_1_BETA. I'm guessing another case of cvs tag missing files that weren't in the checkout. * There are a number of files that lack the REL2_0 tag and REL2_0B branch, though they should have it according to file dates. These appear to be exactly the files that were in the separate documentation repository at the time, so that probably tells us the mechanism for missing them. I wonder if we should consider fixing some or all of these things on the master CVS repository. I wouldn't be too eager to inject those fake .0 commits for fear of breakage, but moving tags to where they ought to have been all along seems like it might be a good thing to do independent of git. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Robert Haas robertmh...@gmail.com writes: Regrettably, all of your attachments came through as part of the actual email, both in my GMail and in the archives. I hate technology. Sorry about that. Here's another try with the stuff in a tarball. This time, I also remembered to include cvs2git.options; although I think it's the same as Max's original except for -r'cvsroot/pgsql', +r'/cvsroot/pgsql', I'll address the other points in a bit. regards, tom lane bind9DKoETd5P.bin Description: conv.tar.gz -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Robert Haas robertmh...@gmail.com writes: On Sun, Sep 12, 2010 at 11:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Having completed that comparison, I then moved on to trying to get rid of the discrepancies in the git conversion; particularly, trying to get rid of the manufactured commits. I didn't have much success in that for the cases where the manufactured commit was caused by a back-branch file addition. [...] We still have manufactured commits either way, but they are just cosmetic so I guess we should live with them. I'm not really following what the history looks like here. What are the contents (git show) of the manufactured commit? A typical example is commit 4d2ac8075a93c685dbbe920f4bac23288dd7cf11 Author: PostgreSQL Daemon webmas...@postgresql.org Date: Tue Nov 22 18:17:36 2005 + This commit was manufactured by cvs2svn to create branch 'REL7_4_STABLE'. Cherrypick from master 2005-11-22 18:17:34 UTC Bruce Momjian br...@momjian.us 'Re-run pgindent, fixing a problem where comment lines after a blank': src/port/unsetenv.c diff --git a/src/port/unsetenv.c b/src/port/unsetenv.c new file mode 100644 index 000..bdfb3f6 --- /dev/null +++ b/src/port/unsetenv.c @@ -0,0 +1,56 @@ + [ entire contents of unsetenv.c here ] In the cases where I inserted a dead .0 revision, this is followed by something like commit a1bdd263ca8ff657365a97a560f6371f39295efc Author: Bruce Momjian br...@momjian.us Date: Tue Nov 22 18:17:37 2005 + Mark branch as deleted. diff --git a/src/port/unsetenv.c b/src/port/unsetenv.c deleted file mode 100644 index bdfb3f6..000 --- a/src/port/unsetenv.c +++ /dev/null @@ -1,56 +0,0 @@ - [ entire contents of unsetenv.c here too ] I'm a bit disappointed by the fact that we get either of these. I had gathered from Max's comments that the dead-revision-at-the-base-of-the- branch trick is considered standard in newer CVS versions, and so I'd hoped that cvs2git would understand the construct and not generate either of these commits. Possibly the hacked-up revisions I inserted are enough different from the regular kind to confuse it. I also found numerous places where we'd been sloppy about placing tags. I wonder if we should consider fixing some or all of these things on the master CVS repository. I wouldn't be too eager to inject those fake .0 commits for fear of breakage, but moving tags to where they ought to have been all along seems like it might be a good thing to do independent of git. Yeah, that's something I was wondering too. Applying these fixes to the master repository would also reduce the number of things we have to remember to do during the final conversion. OTOH, there's that risk of breaking something. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Mon, Sep 13, 2010 at 11:48 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sun, Sep 12, 2010 at 11:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Having completed that comparison, I then moved on to trying to get rid of the discrepancies in the git conversion; particularly, trying to get rid of the manufactured commits. I didn't have much success in that for the cases where the manufactured commit was caused by a back-branch file addition. [...] We still have manufactured commits either way, but they are just cosmetic so I guess we should live with them. I'm not really following what the history looks like here. What are the contents (git show) of the manufactured commit? A typical example is commit 4d2ac8075a93c685dbbe920f4bac23288dd7cf11 Author: PostgreSQL Daemon webmas...@postgresql.org Date: Tue Nov 22 18:17:36 2005 + This commit was manufactured by cvs2svn to create branch 'REL7_4_STABLE'. Cherrypick from master 2005-11-22 18:17:34 UTC Bruce Momjian br...@momjian.us 'Re-run pgindent, fixing a problem where comment lines after a blank': src/port/unsetenv.c diff --git a/src/port/unsetenv.c b/src/port/unsetenv.c new file mode 100644 index 000..bdfb3f6 --- /dev/null +++ b/src/port/unsetenv.c @@ -0,0 +1,56 @@ + [ entire contents of unsetenv.c here ] In the cases where I inserted a dead .0 revision, this is followed by something like commit a1bdd263ca8ff657365a97a560f6371f39295efc Author: Bruce Momjian br...@momjian.us Date: Tue Nov 22 18:17:37 2005 + Mark branch as deleted. If we have two commits one right after the other that cancel each other out, we might be able to write them both out of the history using git-filter-branch. But if Max or Michael can shed any light on why it's happening, that might lead to a simpler solution. I also found numerous places where we'd been sloppy about placing tags. I wonder if we should consider fixing some or all of these things on the master CVS repository. I wouldn't be too eager to inject those fake .0 commits for fear of breakage, but moving tags to where they ought to have been all along seems like it might be a good thing to do independent of git. Yeah, that's something I was wondering too. Applying these fixes to the master repository would also reduce the number of things we have to remember to do during the final conversion. OTOH, there's that risk of breaking something. Hand-written patches that apply directly to the RCS files seem like they'd be a risk for breakage, but I don't see why moving tags around would be all that dangerous, especially in cases where you can do it by running 'cvs' itself rather than 'rcs'. That should just be routine stuff, no? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Robert Haas robertmh...@gmail.com writes: On Mon, Sep 13, 2010 at 11:48 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I wonder if we should consider fixing some or all of these things on the master CVS repository. I wouldn't be too eager to inject those fake .0 commits for fear of breakage, but moving tags to where they ought to have been all along seems like it might be a good thing to do independent of git. Yeah, that's something I was wondering too. Applying these fixes to the master repository would also reduce the number of things we have to remember to do during the final conversion. OTOH, there's that risk of breaking something. Hand-written patches that apply directly to the RCS files seem like they'd be a risk for breakage, but I don't see why moving tags around would be all that dangerous, especially in cases where you can do it by running 'cvs' itself rather than 'rcs'. That should just be routine stuff, no? Hrm, well, keep in mind that most of these problems were *created* by careless use of cvs tag. At the moment I'm leaning towards the idea that we should leave the CVS repository as it is, rather than take any risk of making things worse. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Mon, Sep 13, 2010 at 1:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Sep 13, 2010 at 11:48 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I wonder if we should consider fixing some or all of these things on the master CVS repository. I wouldn't be too eager to inject those fake .0 commits for fear of breakage, but moving tags to where they ought to have been all along seems like it might be a good thing to do independent of git. Yeah, that's something I was wondering too. Applying these fixes to the master repository would also reduce the number of things we have to remember to do during the final conversion. OTOH, there's that risk of breaking something. Hand-written patches that apply directly to the RCS files seem like they'd be a risk for breakage, but I don't see why moving tags around would be all that dangerous, especially in cases where you can do it by running 'cvs' itself rather than 'rcs'. That should just be routine stuff, no? Hrm, well, keep in mind that most of these problems were *created* by careless use of cvs tag. At the moment I'm leaning towards the idea that we should leave the CVS repository as it is, rather than take any risk of making things worse. I think that I have never, and am never likely ever to, hear anyone describe you as careless. I feel pretty much 100% safe having you retag those releases to match the tarballs. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
I wrote:
I'm a bit disappointed by the fact that we get either of these. I had
gathered from Max's comments that the dead-revision-at-the-base-of-the-
branch trick is considered standard in newer CVS versions, and so I'd
hoped that cvs2git would understand the construct and not generate
either of these commits. Possibly the hacked-up revisions I inserted
are enough different from the regular kind to confuse it.
Hah: a bit of digging in the cvs2svn sources found this:
def _is_unneeded_initial_branch_delete(self, lod_items, metadata_db):
Return True iff the initial revision in LOD_ITEMS can be deleted.
if not lod_items.cvs_revisions:
return False
cvs_revision = lod_items.cvs_revisions[0]
if cvs_revision.ntdbr:
return False
if not isinstance(cvs_revision, CVSRevisionAbsent):
return False
if cvs_revision.branch_ids:
return False
log_msg = metadata_db[cvs_revision.metadata_id].log_msg
return bool(re.match(
r'file .* was added on branch .* on '
r'\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2}( [\+\-]\d{4})?'
'\n$',
log_msg,
))
So it looks like I have to make the dead revisions' log messages match
that regexp. Off to make another try.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
I wrote:
return bool(re.match(
r'file .* was added on branch .* on '
r'\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2}( [\+\-]\d{4})?'
'\n$',
log_msg,
))
So it looks like I have to make the dead revisions' log messages match
that regexp. Off to make another try.
It works! Now I don't see either the manufactured commits or the
patched-in deletions.
I had not previously bothered to patch the places where a file was added
on the branch immediately after being added on the main, but now it
seems worth doing. That will get us down to a *very* small number of
manufactured commits in the final version.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
On Mon, Sep 13, 2010 at 21:28, Tom Lane t...@sss.pgh.pa.us wrote:
I wrote:
return bool(re.match(
r'file .* was added on branch .* on '
r'\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2}( [\+\-]\d{4})?'
'\n$',
log_msg,
))
So it looks like I have to make the dead revisions' log messages match
that regexp. Off to make another try.
It works! Now I don't see either the manufactured commits or the
patched-in deletions.
I had not previously bothered to patch the places where a file was added
on the branch immediately after being added on the main, but now it
seems worth doing. That will get us down to a *very* small number of
manufactured commits in the final version.
That's awesome!
Thanks so much for doing this. I've come to realize I know far too
little about *cvs* to work on those things myself :S
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Tom Lane wrote: the tarball was actually made. In particular, the tags REL6_5, REL7_1, and REL7_1_2 don't match the tarballs they ought to. I don't have a whole lot of faith in some of the other early tags either, because we don't seem to have an archived tarball to compare them to. I believe I have those on a CDROM here. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Bruce Momjian br...@momjian.us writes: Tom Lane wrote: the tarball was actually made. In particular, the tags REL6_5, REL7_1, and REL7_1_2 don't match the tarballs they ought to. I don't have a whole lot of faith in some of the other early tags either, because we don't seem to have an archived tarball to compare them to. I believe I have those on a CDROM here. If you can recover any of the releases that aren't on ftp-archive, please send me copies. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Report: removing the inconsistencies in our CVS-git conversion
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Tom Lane wrote: the tarball was actually made. In particular, the tags REL6_5, REL7_1, and REL7_1_2 don't match the tarballs they ought to. I don't have a whole lot of faith in some of the other early tags either, because we don't seem to have an archived tarball to compare them to. I believe I have those on a CDROM here. If you can recover any of the releases that aren't on ftp-archive, please send me copies. Sure. I have a copy of our ftp site /pub as of 6.3 and have put it online: http://momjian.us/expire/pgsql_ftp_6.3/ Unfortunately I don't see anything there that isn't already here: ftp://ftp-archives.postgresql.org/pub/source/ but let me know if you find something new. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
