I've attached an improved version of the Subversion to Fossil script in the Cookbook: http://www.fossil-scm.org/fossil/wiki?name=Cookbook#SVN

The general mode of operation is the same, but I've made a bunch of improvements. In rough order of importance:

- The most important change is that it's thoroughly protected against overwriting data when run multiple times. If you run it twice in direct succession, nothing happens the second time. If you remove some intermediary data created during a prior run and run the script again, it will recreate that intermediary *only*.

The idea is that a conversion often doesn't go 100% correct the first time, but you don't need to do the entire thing over to fix the problem. So, you fix the one problem and restart, then test again, iterating your way toward a correct conversion.

- Added the 'fakeauthor' hook. In my repository, I found a few svn "authors" that were essentially bogus, and didn't want to give them Fossil user logins. The script looks like this:

    echo "Fred Flintstone <f...@example.com>"

That is, this step documents that Fred owns checkins no one else claims, but we've minimized our repo's attack surface by not creating a repo login for him.

If you find yourself needing the fakeauthor hook, you're probably going to want to edit authors.txt, removing bogus users. (And since the new script never overwrites an existing authors.txt, you don't have to keep making these edits if you're still iterating your way toward a correct conversion.)

- Added the 'git-fixups' hook. This lets you write a script that fixes things in the intermediate Git stage before pouring that into Fossil. I've attached a somewhat sanitized version of my script.

I would like someone clueful in the way the Git to Fossil step works to check the second and third stages of that hook script. I found them on a web page and added them to my script. I have no idea if they're actually useful in the context of a Fossil conversion, or if Fossil already effectively does these things when importing from Git. If they're useless, please remove them.

- Passing --stdlayout to "git svn clone". That makes it assume the standard "trunk, branches, tags" svn repo structure, so the branch and tag data gets preserved in Fossil.

- Passing -q to same, to reduce the noisiness.

- Removed --full-tree from "git fast-export", since it doesn't seem to improve the conversion quality, and it makes the conversion take considerably longer. (As in many hours vs. 10 minutes!)

- Instead of creating an intermediary users.txt file, it parses the data out of authors.txt. DRY principle.

- It no longer takes an admin user parameter. I think the default Fossil behavior of using your username makes more sense. It could easily be put back in, if someone disagrees, though if so, it should probably be last, and optional.

- Now that your username is assumed to be the admin user, and your user probably owns checkins in the svn repo, we have a near-certainty of an attempt to create a duplicate user in the Fossil repo. The existing cookbook script simply skips that user when that happens. My script recognizes this situation and assumes you want to give the user Setup permissions, and override the default random password.

- The script tries to detect if X is available and runs "fossil ui" only if it thinks it will succeed. If not, it runs "fossil server" instead.

This is why the previous change had to be made, by the way: the original script assumed you could do initial user setup via "fossil ui", since it makes you a superuser, but that doesn't work if you're doing the conversion on a remote box and forwarding X11 over SSH. So, we need a user with Setup privilege and a known password so we can do this initial setup remotely.

(Yes, I'm aware that the random password is written to the console before all this. That means I have to keep copy-and-pasting the password on each conversion iteration, instead of letting the browser remember the hardcoded myuser1234 password. I don't do any password hardening until the conversion is working correctly.)

- Added a bunch of time(1) commands, so I can monitor the effect of my changes to the script. They could be removed now, but I think it's interesting to see how long the various steps take.

- Assorted output message improvements.

It wouldn't break my heart if the current Cookbook recipe were replaced with this improved version. :)
# Based on http://www.fossil-scm.org/fossil/wiki?name=Cookbook#SVN

# parameters:
# 1 - URL of Subversion repository to convert (e.g. file:///svn/foo)
# 2 - name of Fossil repo file (e.g. myrepo.fossil)

# NOTE: The repository will be created in the current directory if
#       you don't give a path in the second argument!

die() {
        echo $1
        rm -rf $tmpgit
        exit 1

if [ "$2" == "" ]
        echo "Syntax:"
        echo "  svn2fossil svn-repo-url fossil-repo-name"
        die "Please make sure you have all three parameters set!"

tmpgit=/tmp/`basename $fossil | sed -re 's/\.[a-z]+$//'`-tmp-git

if [ ! -e $authors ]
        echo "Getting user names..."

        svn log $svnurl |
                egrep '^r[0-9]+ \| ' |
                grep -v '(no' |
                cut -f3 -d' ' |
                sort -u |
                sed -e 's/.*/& = & <&>/' > $authors ||
                die "Failed to convert Subversion user list to Git authors 

if [ ! -d "$tmpgit/.git" ]
        echo "Importing from SVN to Git.  This will take some time, be patient!"

        time git svn clone -q --stdlayout --no-metadata \
                        --authors-file=$authors \
                        --authors-prog=./fakeauthor \
                        $svnurl $tmpgit ||
                die "Failed creating Git clone of Subversion repository."

        if [ -x git-fixups ] ; then ./git-fixups $tmpgit ; fi

if [ -e $fossil ]
        echo "$fossil already exists; will not overwrite it."
        echo "If you mean to replace it, remove it and restart this script."
        exit 1

        echo "Importing from Git to Fossil.  This will take some time..."
        cd $tmpgit
        time git fast-export --all | fossil import --git $fossil ||
                die "Failed to import Git repository into Fossil."

echo "Adding users to Fossil repository."
while read line
        user=`echo $line | cut -f1 -d '=' | sed -e 's/ *$//'`
        email=`echo $line | cut -f2 -d '=' | sed -e 's/^ *//'`
        if fossil user new $user "$email" "${user}1234" -R $fossil
                fossil user capabilities $user 'v' -R $fossil
                # Presumably this is the default admin user, so just change
                # its password and permissions.
                fossil user capabilities $user 's' -R $fossil
                fossil user password $user "${user}1234" -R $fossil
done < $authors

if [ -n "$DISPLAY" ] && xterm -e exit
        echo "Check out the conversion in the Fossil UI, then Ctrl-C when done."
        fossil ui $fossil
        echo "X isn't available, so serving $fossil on"
        echo "http://`hostname`:8080/  Check it out, then Ctrl-C when done."
        fossil serve $fossil

echo "You can rm -rf $tmpgit if you're happy with that result."
## Remove branches and tags we don't really need.  These are all
## historical things that were directly replaced by newer names.
if [ \( -n "$1" \) -a \( -d "$1" \) ] ; then cd "$1" ; fi
for b in \
        mms6.3 mms6.3@7569 mms6.3@9526 \
        tags/mms5_00-alpha5 tags/mms5_00-alpha6
        git branch -D -r "$b"

## Clean up useless commits created by conversion steps.  This step and
## the next are from http://thomasrast.ch/git/git-svn-conversion.html
echo Cleaning up Git clone...
time git filter-branch --prune-empty --tag-name-filter cat -- --all

## Change tag commits to actual Git tags
echo Converting Subversion tag commits to Git tags...
time git for-each-ref --format="%(refname)" refs/remotes/tags/ |
while read tag
        GIT_COMMITTER_DATE="$(git log -1 --pretty=format:"%ad" "$tag")" \
        GIT_COMMITTER_EMAIL="$(git log -1 --pretty=format:"%ce" "$tag")" \
        GIT_COMMITTER_NAME="$(git log -1 --pretty=format:"%cn" "$tag")" \
        git tag -m "$(git for-each-ref --format="%(contents)" "$tag")" \
                                ${tag#refs/remotes/tags/} "$tag"
echo "Fred Flintstone <f...@example.com>"
fossil-users mailing list

Reply via email to