Re: upgrading server
On Tue, Jul 25, 2017 at 9:00 PM, Mark Phippardwrote: > On Tue, Jul 25, 2017 at 2:00 PM, Andy So wrote: >> >> We have an old subversion version 1.4.3 (r23084) running on Solaris. >> >> We would like to upgrade to use new hardware on Linux based OS (CentOS >> 6.9), possibly version 1.8.x or 1.9.x >> >> >> >> Our plan is to installed and configure the latest SVN on CentOS 6.9. Then >> go through dump and load of the repository as described in various online >> post and documentations. The repository is quite large…guessing the size to >> be in the order of 20-40GB >> >> >> >> Before we start undertaking such tasks >> >> 1. Does anyone know if there are there any problem/gotcha in >> migrating the repository? >> >> 2. Does anyone know how long it would take to export the repository >> of this size? This will give us an estimate how long to schedule down time >> and cut off time. >> >> >> >> Thanks for any insight. > > > > Dump and load is a good idea because it lets the repository be rewritten > using the newer code and repository format which will give you a smaller > repository that will run a little faster. That said you do not HAVE to > dump/load. You have other options: > > 1. Just move the repository folder to the new server. Perhaps using tar and > then moving the archive. > > 2. Instead of using dump/load, use svnsync. This gives all the benefits of > the dump/load but allows you to shrink your downtime to almost nothing. > Just svnsync the repository to the new server. This will probably take a > long time, but it does not matter since the original server can be running > while it happens. At the time of your choosing, do a final svnsync, and > then shutdown the old server and use the new one. > > 3. Do option 1 now and then do a dump/load or svnsync at some future time > that is more convenient for downtime. It will probably run faster since it > is on new and better hardware too. > > There are some gotchas no matter what: > > 1. Does the new server have a new hostname or do you intend to update DNS > to point to new server? If you are not doing the latter, then all of your > existing working copies and scripts have to be modified for the new server. > Also any use of svn:externals property has to be modified. > > 2. With an old repository there is a good chance you will run into bugs in > your data that cause svnsync or load to fail. There are workaround for > different failures but be prepared to run into them and account for finding > the solutions. SVN 1.8 and 1.9 have added various options to let you > workaround some of the known bugs. A common problem is having svn: > properties that are not UTF-8 encoded or do not have LF line endings. > svnsync has this option to work around the encoding problem: > > --source-prop-encoding ARG : convert translatable properties from encoding > ARG > to UTF-8. If not specified, then properties are > presumed to be encoded in UTF-8. > > And it automatically fixes the LF problems. > > svnadmin load does not have any options to fix the problems, but you can add > the --bypass-prop-validation option to ignore them and just carry the > problems into your new repository. Indeed, dump/load is a bit more limited than svnsync regarding normalization. Also, be careful: both dump/load and svnsync do not migrate your hook scripts, config files and locks (server side locks of the svn:needs-lock type). I've just extended the FAQ entry about dump/load with some of the known caveats, and explained a procedure to do dump/load with minimal downtime. See: http://subversion.apache.org/faq.html#dumpload -- Johan
Re: upgrading server
On Tue, Jul 25, 2017 at 2:00 PM, Andy Sowrote: > We have an old subversion version 1.4.3 (r23084) running on Solaris. > > We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9), > possibly version 1.8.x or 1.9.x If you're bumping Subversion release to 1.8.x or 1.9.x, I'd urge you to go to CentOS 7, which at least starts with Subverison 1.7, and get the latest RPM's from Wandisco. > Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go > through dump and load of the repository as described in various online post > and documentations. The repository is quite large…guessing the size to be > in the order of 20-40GB See above. Installing the latest Subversion on CentOS 6 gets into some fairly awkward library dependencies, especially for the "serf" libraries which are notably out of date on CentOS 6. > Before we start undertaking such tasks > > 1. Does anyone know if there are there any problem/gotcha in migrating > the repository? > > 2. Does anyone know how long it would take to export the repository of > this size? This will give us an estimate how long to schedule down time and > cut off time. Definitely use svnsync, if possible, so you have no downtime and can write-lock the original repository and do a DNS based switchover. Rollback, if there are *any* mismatched writes to the new repository, means starting over with all working copies, which can be quite painful. > Thanks for any insight. You might consider less of an extreme version update. The more versions you update between, I've noted more likelihood of incompatible workflows such as the changes in the "svn:external" formats.
Re: upgrading server
Don't forget that svnadmin dump/load and svnsync don't preserve hook scripts or other ancillary data that might be in your repository directory on the server, so if you have hook scripts, authorization rules or other customizations, carry those over to the new server manually. And of course any configuration files for httpd or svnserve.
RE: upgrading server
>> Does anyone know how long it would take to export the repository of this >> size? This will give us an estimate how long to schedule down time and cut >> off time. Svnsync is the easy option. If you insist on doing a dump/load, then a) you can time a test run of a dump/load, and b) "svnadmin dump" allows you to specify revision ranges which means you can do incremental dumps. You can dump/load the bulk of the repo now and then during the migration, run another dump/load to catch any new commits. http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate The only caveat I can think of is if someone changes revision properties (e.g. the commit message) between the time of the initial dump and the migration. But you can track/prevent those with a hook script. (And is another reason why svnsync is preferred.)
Re: upgrading server
On 7/25/2017 11:00 AM, Andy So wrote: We have an old subversion /version 1.4.3 (r23084) /running on Solaris. We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9), possibly version 1.8.x or 1.9.x Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go through dump and load of the repository as described in various online post and documentations. The repository is quite large…guessing the size to be in the order of 20-40GB Before we start undertaking such tasks 1.Does anyone know if there are there any problem/gotcha in migrating the repository? 2.Does anyone know how long it would take to export the repository of this size? This will give us an estimate how long to schedule down time and cut off time. Thanks for any insight. I recommend CentOS 7.x; CentOS 6.x is nearing the end of its support lifetime. I also installed an SSD on my new machine - much faster than rotating media, and Subversion's "write once" philosophy (old revisions are essentially immutable) works well on an SSD. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
Re: upgrading server
On Tue, Jul 25, 2017 at 2:00 PM, Andy Sowrote: > We have an old subversion *version 1.4.3 (r23084) *running on Solaris. > > We would like to upgrade to use new hardware on Linux based OS (CentOS > 6.9), possibly version 1.8.x or 1.9.x > > > > Our plan is to installed and configure the latest SVN on CentOS 6.9. Then > go through dump and load of the repository as described in various online > post and documentations. The repository is quite large…guessing the size > to be in the order of 20-40GB > > > > Before we start undertaking such tasks > > 1. Does anyone know if there are there any problem/gotcha in > migrating the repository? > > 2. Does anyone know how long it would take to export the repository > of this size? This will give us an estimate how long to schedule down time > and cut off time. > > > > Thanks for any insight. > Dump and load is a good idea because it lets the repository be rewritten using the newer code and repository format which will give you a smaller repository that will run a little faster. That said you do not HAVE to dump/load. You have other options: 1. Just move the repository folder to the new server. Perhaps using tar and then moving the archive. 2. Instead of using dump/load, use svnsync. This gives all the benefits of the dump/load but allows you to shrink your downtime to almost nothing. Just svnsync the repository to the new server. This will probably take a long time, but it does not matter since the original server can be running while it happens. At the time of your choosing, do a final svnsync, and then shutdown the old server and use the new one. 3. Do option 1 now and then do a dump/load or svnsync at some future time that is more convenient for downtime. It will probably run faster since it is on new and better hardware too. There are some gotchas no matter what: 1. Does the new server have a new hostname or do you intend to update DNS to point to new server? If you are not doing the latter, then all of your existing working copies and scripts have to be modified for the new server. Also any use of svn:externals property has to be modified. 2. With an old repository there is a good chance you will run into bugs in your data that cause svnsync or load to fail. There are workaround for different failures but be prepared to run into them and account for finding the solutions. SVN 1.8 and 1.9 have added various options to let you workaround some of the known bugs. A common problem is having svn: properties that are not UTF-8 encoded or do not have LF line endings. svnsync has this option to work around the encoding problem: --source-prop-encoding ARG : convert translatable properties from encoding ARG to UTF-8. If not specified, then properties are presumed to be encoded in UTF-8. And it automatically fixes the LF problems. svnadmin load does not have any options to fix the problems, but you can add the --bypass-prop-validation option to ignore them and just carry the problems into your new repository. -- Thanks Mark Phippard http://markphip.blogspot.com/