Re: State of CVS-to-git conversion tools (Was: Re: cvsps: bad usage: invalid argument --norc)
Apologies for the somewhat belated reply. I've been even busier than usual lately and am about to be traveling for a week. Ilya Basin basini...@gmail.com: Hi Eric. ESR cvs-fast-export does not have incremental-import support. ESR Whether git-cvs-import has it depend on which version you have ESR and what backend it it is using. I don't maintain that wrapper. Did you mean git-fast-import? Or do you know any wrapper that already uses cvsps3 --fast-export? No, I meant git-cvs-import. I wrote a version of it that supports cvsps3, but Junio chose to keep the old wrapper. Apparently he would rather inflict cvsps2's rather serious known bugs on users than break backward compatibility even a little. First of all, I think cvsps3 has almost everithing required for incremental import: one could just take the date of the last commit and invoke cvs ps with the '-d' flag. However, to import new commits into existing branches the stream should contain the from command in oldest commits in each branch (now missing). If the branch already exists in the target git repo, it's easy to refer it in the stream: from refs/heads/branchname^0 Look at the -i option. That may do what you need. But if the branch is new, but it's parent commit is already imported, I guess, the only way to refer it is by its SHA-1 Eric, what parent information can cvsps provide for the first commit in a branch, when invoked with the '-d' flag? At the moment it doesn't provide any at all. That case wasn't on my radar when I was fixing the code. If you can specify a behavior you think would be useful, I'm listening. -- a href=http://www.catb.org/~esr/;Eric S. Raymond/a -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re[2]: State of CVS-to-git conversion tools (Was: Re: cvsps: bad usage: invalid argument --norc)
But if the branch is new, but it's parent commit is already imported, I guess, the only way to refer it is by its SHA-1 Eric, what parent information can cvsps provide for the first commit in a branch, when invoked with the '-d' flag? ESR At the moment it doesn't provide any at all. That case wasn't on my ESR radar when I was fixing the code. If you can specify a behavior you ESR think would be useful, I'm listening. For new branches the 'from' command can refer the common ancestor in an existing branch. For example: /--E thebranch / A---B---C---D master Commit E is newer than D; we already imported D; thebranch is new. Instead of: from refs/heads/thebranch^0 refer the parent as: from refs/heads/master^2 ESR Look at the -i option. That may do what you need. ESR OK, something's wrong with the man page: starting with '-A' the description is unstructured: -A authormap Apply an author-map file to the attribution lines. Each line must be of the form ferd = Ferd J. Foonly foo...@foo.com America/Chicago and will be applied to map the Unix username ferd to the DVCS-style user identity specified after the equals sign. The timezone field (after and whitespace) is optional and (if present) is used to set the timezone offset to be attached to the date; acceptable formats for the timezone field are anything that can be in the TZ environment variable, including a [+-]hhmm offset. Whitespace around the equals sign is stripped. Lines beginning with a # or not containing an equals sign are silently ignored. -R revmap:: Write a revision map to the specified argument filename. Each line of the revision map consists of three whitespace-separated fields: a filename, an RCS revision number, and the mark of the commit to which that filename-revision pair was assigned. -v:: show very verbose parsing messages. -t:: show some brief memory usage statistics. --summary-first:: when multiple patchset diffs are being generated, put the patchset summary for all patchsets at the beginning of the output. --diffs-opts option string:: send a custom set of options to diff, for example to increase the number of context lines, or change the diff format. --debuglvl bitmask:: enable various debug output channels. -Z compression:: A value 1-9 which specifies amount of compression. A value of 0 disables compression. --root cvsroot:: Override the setting of CVSROOT (overrides working directory and environment). -i:: Incremental export. Each commit with no ancestor gets a from pointer name. When importing to an existing repository, this will attach each such commit as a child of the last commit on $BRANCH in the existing repository. -k:: Kill keywords: will extract files with -kk from the CVS archive to avoid noisy changesets. -T:: Force deterministic dates for regression testing. Each patchset will have a monotonic-increasing attributed date computed from its patchset ID. --fast-export:: Emit the report as a git import stream. --convert-ignores:: Convert ..cvsignore files to .gitignore files. --reposurgeon:: Emit for each commit a list of the CVS file:revision pairs composing it as a bzr-style commit property named cvs-revisions. From version 2.12 onward, reposurgeon can interpret these and use them as hints for reference-lifting. -V:: Emit the program version and exit. module-path:: Operate on the specified module. If this option is not given, either the CVSROOT environment variable must be set to point directly at the module or cvsps must be run in a checkout directory or repository module subdirectory. -- -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: State of CVS-to-git conversion tools (Was: Re: cvsps: bad usage: invalid argument --norc)
Ilya Basin basini...@gmail.com: For new branches the 'from' command can refer the common ancestor in an existing branch. For example: /--E thebranch / A---B---C---D master Commit E is newer than D; we already imported D; thebranch is new. Instead of: from refs/heads/thebranch^0 refer the parent as: from refs/heads/master^2 Understood. Do you actually need this much generality in practice, or is it a theoretical case? OK, something's wrong with the man page: starting with '-A' the description is unstructured: Interesting. The aciidoc parser got a little confused, but inserting some blank lines fixed it. -- a href=http://www.catb.org/~esr/;Eric S. Raymond/a -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re[2]: State of CVS-to-git conversion tools (Was: Re: cvsps: bad usage: invalid argument --norc)
ESR Ilya Basin basini...@gmail.com: For new branches the 'from' command can refer the common ancestor in an existing branch. For example: /--E thebranch / A---B---C---D master Commit E is newer than D; we already imported D; thebranch is new. Instead of: from refs/heads/thebranch^0 refer the parent as: from refs/heads/master^2 ESR Understood. Do you actually need this much generality in practice, ESR or is it a theoretical case? Such cases occur quite often. -- -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re[2]: State of CVS-to-git conversion tools (Was: Re: cvsps: bad usage: invalid argument --norc)
Hi Eric. ESR cvs-fast-export does not have incremental-import support. ESR Whether git-cvs-import has it depend on which version you have ESR and what backend it it is using. I don't maintain that wrapper. Did you mean git-fast-import? Or do you know any wrapper that already uses cvsps3 --fast-export? I need it, because full import takes too long. The central repo of my employer is CVS, other people commit to it and I use git internally to be able to tidy my commit history before exporting to CVS. ESR You are out of luck. That feature was dependent on a very fragile ESR coupling... OK, OK, I get it. First of all, I think cvsps3 has almost everithing required for incremental import: one could just take the date of the last commit and invoke cvs ps with the '-d' flag. However, to import new commits into existing branches the stream should contain the from command in oldest commits in each branch (now missing). If the branch already exists in the target git repo, it's easy to refer it in the stream: from refs/heads/branchname^0 But if the branch is new, but it's parent commit is already imported, I guess, the only way to refer it is by its SHA-1 Eric, what parent information can cvsps provide for the first commit in a branch, when invoked with the '-d' flag? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: State of CVS-to-git conversion tools (Was: Re: cvsps: bad usage: invalid argument --norc)
Ilya Basin basini...@gmail.com: Hi Eric. I tried --fast-export. It's 2 times faster. The first thing that differs: in cvsps2 commits with adjacent timestamps were joined into one (see the attached files). Do you know the reason? The cvsps guy included code to do that. Keith Packard didn't. Sorry I can't be more helpful, but that's about all I know. I didn't write either analysis stage; I understand cvsps's, somewhat, because I had to fix several nasty bugs in it. I *don't* understand cvs-fast-export's analysis stage very well yet, because it has no obvious bugs that have required me to dive in. (Keith's notes document one major bug, which may be inherent to the mismatch between file- and changest-orientation and not fixable in the general case, though I will try.) Does this --fast-export thing support what John mentioned, the incremental import support? Does 'git fast-import' has it? cvs-fast-export does not have incremental-import support. Whether git-cvs-import has it depend on which version you have and what backend it it is using. I don't maintain that wrapper. I need it, because full import takes too long. The central repo of my employer is CVS, other people commit to it and I use git internally to be able to tidy my commit history before exporting to CVS. You are out of luck. That feature was dependent on a very fragile coupling between the old output format and a bunch of unmaintainably horrible Perl in the git-cvs-import wrapper script. It didn't work very well; frankly, I'm amazed it worked at all. The things I had to do to fix the serious bugs in cvsps2 and make it output a fast-import stream had the side effect of breaking that coupling. cvsps3 won't give you that feature. Dropping back to cvsps2 to keep that feature will expose you to the cvsps2 bugs. I'm sorry these tools are such a mess. I'm trying to fix that, but it's hard, slow work. The problems are deeply ugly and the edge cases have poisoned spikes. -- a href=http://www.catb.org/~esr/;Eric S. Raymond/a -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: State of CVS-to-git conversion tools (Was: Re: cvsps: bad usage: invalid argument --norc)
Hi Eric. I tried --fast-export. It's 2 times faster. The first thing that differs: in cvsps2 commits with adjacent timestamps were joined into one (see the attached files). Do you know the reason? Does this --fast-export thing support what John mentioned, the incremental import support? Does 'git fast-import' has it? I need it, because full import takes too long. The central repo of my employer is CVS, other people commit to it and I use git internally to be able to tidy my commit history before exporting to CVS. -- cvsps2: commit c0654845791da1eefb1e223c8001169d61f9de51 Author: reevzh1 reevzh1 Date: Fri Jul 28 10:47:17 2006 + no message components/rapportlets/src/com/sicap/guif/portalframework/portlets/rap/ReportFraudCDRQuerierPortlet.java | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) commit 2ed1c92eec518a5752277a16cf2aea6bd488d6be Author: reevzh1 reevzh1 Date: Fri Jul 28 04:51:06 2006 + no message components/rapportlets/src/com/sicap/guif/portalframework/portlets/rap/CommonAttributeNames.java | 6 ++ components/rapportlets/src/com/sicap/guif/portalframework/portlets/rap/ReportFraudCDRQuerierItem.java | 129 components/rapportlets/src/com/sicap/guif/portalframework/portlets/rap/ReportFraudCDRQuerierPortlet.java | 101 +++ components/rapportlets/src/com/sicap/guif/portalframework/portlets/rap/containers/ReportContainer.java | 48 + resources/ReportFraudCDRQuerier_en.properties | 2 + web/jsp/report_fraudcdr_querier.jsp | 21 ++-- web/jsp/report_fraudcdr_querier_print.jsp | 309 + web/resources/Language_en.properties | 8 +- 8 files changed, 595 insertions(+), 29 deletions(-) commit d6a7335ea24323509df02b08f52af01926eb5e4d Author: redane1 redane1 Date: Thu Jul 6 10:28:24 2006 + no message ... cvsps3: commit d3732dbdf41b348f0050e888733ce2afec05f7f8 Author: reevzh1 reevzh1 Date: Fri Jul 28 10:47:17 2006 + no message components/rapportlets/src/com/sicap/guif/portalframework/portlets/rap/ReportFraudCDRQuerierPortlet.java | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) commit 158bc55affa80b60471ea6026b1393934f928e58 Author: reevzh1 reevzh1 Date: Fri Jul 28 04:51:22 2006 + no message components/rapportlets/src/com/sicap/guif/portalframework/portlets/rap/ReportFraudCDRQuerierItem.java | 129 + components/rapportlets/src/com/sicap/guif/portalframework/portlets/rap/containers/ReportContainer.java | 48 ++ web/jsp/report_fraudcdr_querier_print.jsp | 309 +++ web/resources/Language_en.properties | 8 +- 4 files changed, 493 insertions(+), 1 deletion(-) commit 1ba90a2baebee9f1838dab04f129c7704331e0a8 Author: reevzh1 reevzh1 Date: Fri Jul 28 04:51:06 2006 + no message components/rapportlets/src/com/sicap/guif/portalframework/portlets/rap/CommonAttributeNames.java | 6 components/rapportlets/src/com/sicap/guif/portalframework/portlets/rap/ReportFraudCDRQuerierPortlet.java | 101 + resources/ReportFraudCDRQuerier_en.properties | 2 ++ web/jsp/report_fraudcdr_querier.jsp | 21 4 files changed, 102 insertions(+), 28 deletions(-) commit d6a7335ea24323509df02b08f52af01926eb5e4d Author: redane1 redane1 Date: Thu Jul 6 10:28:24 2006 + no message ...
Re: State of CVS-to-git conversion tools (Was: Re: cvsps: bad usage: invalid argument --norc)
Ilya Basin wrote: Does this --fast-export thing support what John mentioned, the incremental import support? Does 'git fast-import' has it? I need it, because full import takes too long. The central repo of my employer is CVS, other people commit to it and I use git internally to be able to tidy my commit history before exporting to CVS. Errr... the need to interact with *live* CVS repository is reason behind wanting incremental import, isn't it? I wonder how hard would be to create a remote helper for CVS, so that one could treat CVS repository as foreign remote, fetching from it and pushing to it... BTW. can you convince your employer to move to more modern version control system (be it Subversion, Kiln Harmony, Veracity, Bazaar, Mercurial or Git) rather than rely on system which had last release in 2008 i.e. 4 years ago, doesn't support atomic commits, doesn't support versioning of whole repository aka. changesets (beside tags), has slow and complicated branching and even more complicated merging, etc.? http://stackoverflow.com/questions/802573/difference-between-git-and-cvs/824241#824241 -- Jakub Narębski -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
cvsps: bad usage: invalid argument --norc
Hi esr. In cvsps 3.10 the flag --norc was removed. It broke 'git cvsimport'. Please give the option back and write something in the man page like: This option has no effect; it is present for compatibility -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cvsps: bad usage: invalid argument --norc
IB Hi esr. IB In cvsps 3.10 the flag --norc was removed. It broke 'git cvsimport'. IB Please give the option back and write something in the man page like: IB This option has no effect; it is present for compatibility Looks like the tool is completely different. I think I'll have to downgrade. -- -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cvsps: bad usage: invalid argument --norc
On Sun, Apr 14, 2013 at 12:48:00PM +0400, Ilya Basin wrote: IB Hi esr. IB In cvsps 3.10 the flag --norc was removed. It broke 'git cvsimport'. IB Please give the option back and write something in the man page like: IB This option has no effect; it is present for compatibility Looks like the tool is completely different. I think I'll have to downgrade. If you want incremental import support, then that's the best thing to do. There was some discussion about this in January [1] and it doesn't look like cvsps-3 has changed since then, so if you want to incrementally update a Git clone of a CVS repository then cvsps-2 is really the only way to go. If you're doing a one-off import then I recommend ignoring git-cvsimport and investigating cvs2svn[2] and cvs-fast-export [3]. [1] http://thread.gmane.org/gmane.comp.version-control.git/214258/focus=214305 [2] http://cvs2svn.tigris.org/cvs2git.html [3] https://gitorious.org/cvs-fast-export -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
State of CVS-to-git conversion tools (Was: Re: cvsps: bad usage: invalid argument --norc)
Ilya Basin basini...@gmail.com: IB Hi esr. IB In cvsps 3.10 the flag --norc was removed. It broke 'git cvsimport'. IB Please give the option back and write something in the man page like: IB This option has no effect; it is present for compatibility Looks like the tool is completely different. I think I'll have to downgrade. Or you could just use 3.x directly rather than through the git-cvsimport wrapper. It works better that way - it actually ships a fast-import stream that git fast-import can consume directly. Old cvsps (2.x) was very, very broken; there was a bug in it that pretty much guaranteed incorrect conversion of branchy repos. I've fixed that particular bug, and several other serious ones, along with adding the import-stream output stage, but I don't really trust the cvsps code; I think its algorithmic core is weak and klugey and only works semi-by-accident. I do *not* recommend downgrading, it will pitch you from a bad situation into a worse one. Yes, Junio is still shipping a wrapper for 2.x, but that was very much against my advice. I'm also now the maintainer of cvs-fast-export, which used to be Keith Packard's utility for lifting the X.org CVS repo to git. That was before I resurrected it and added a fast-import stream output stage. I *think* cvs-fast-export's algorithms are more correct than cvsps's, but I have not yet been able to pry loose the time to write the really rigorous test suite to verify this. That goal has been second on my prority list for a couple of months now; I keep wanting to get to it but having to fight fires on other projects instead. I wish I could point you at a convertor I really trust. I can't. There is a third tool, cvs2git (based on the analyzer from cvs2svn) that I don't maintain, which has problems of its own. And those three are about it. Yes, it's a swamp. The relatively poor capability of the tools isn't anybody's fault; the problem domain is *nasty*. I've been working a closely related but easier one (Subversion stream dump analysis) for a couple years now and understanding it doesn't make it less ugly. I think with 4-6 weeks of concentrated attention I could clean up the mess and deliver a really high-quality converter, and I'm motivated to do this because I want it as a CVS front end for reposurgeon. But it hasn't happened yet and the incompleteness of my test suite is a blocker in the way. The topo analysis code in all these tools is really fragile and tends to break on old edge cases when you try to teach it to handle new ones, so a good set of regression tests is especially important. And doesn't yet exist, though I have built a decent start for cvsps based on the tests in the git tree. Do you have enough interest and spare cycles to help finish the test suite? Another pair of hands on it might speed things up a lot. -- a href=http://www.catb.org/~esr/;Eric S. Raymond/a -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: State of CVS-to-git conversion tools (Was: Re: cvsps: bad usage: invalid argument --norc)
On 04/14/2013 01:33 PM, Eric S. Raymond wrote: [...] I wish I could point you at a convertor I really trust. I can't. There is a third tool, cvs2git (based on the analyzer from cvs2svn) that I don't maintain, which has problems of its own. ^ Again I request that instead of spreading non-specific FUD about cvs2git, you describe concrete problems so that I can improve the tool. So far about the only problems that you have mentioned that have not been addressed have been stylistic trivia like that you object that cvs2git writes its outputs by pass, to two separate files, rather than writing all of its output to stdout, and that you are bothered that it names its temporary directory cvs2svn-tmp rather than cvs2git-tmp. I would like to fix any conversion deficiencies you can find. I stand behind this tool and think it is the best one out there for extracting history from CVS. Michael -- Michael Haggerty mhag...@alum.mit.edu http://softwareswirl.blogspot.com/ -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
