Re: upgrading server

[Johan Corveleyn]

On Tue, Jul 25, 2017 at 9:00 PM, Mark Phippard  wrote:
> On Tue, Jul 25, 2017 at 2:00 PM, Andy So  wrote:
>>
>> We have an old subversion version 1.4.3 (r23084) running on Solaris.
>>
>> We would like to upgrade to use new hardware on Linux based OS (CentOS
>> 6.9), possibly version 1.8.x or 1.9.x
>>
>>
>>
>> Our plan is to installed and configure the latest SVN on CentOS 6.9. Then
>> go through dump and load of the repository as described in various online
>> post and documentations.  The repository is quite large…guessing the size to
>> be in the order of 20-40GB
>>
>>
>>
>> Before we start undertaking such tasks
>>
>> 1.  Does anyone know if there are there any problem/gotcha in
>> migrating the repository?
>>
>> 2.  Does anyone know how long it would take to export the repository
>> of this size?  This will give us an estimate how long to schedule down time
>> and cut off time.
>>
>>
>>
>> Thanks for any insight.
>
>
>
> Dump and load is a good idea because it lets the repository be rewritten
> using the newer code and repository format which will give you a smaller
> repository that will run a little faster.  That said you do not HAVE to
> dump/load.  You have other options:
>
> 1. Just move the repository folder to the new server.  Perhaps using tar and
> then moving the archive.
>
> 2. Instead of using dump/load, use svnsync.  This gives all the benefits of
> the dump/load but allows you to shrink your downtime to almost nothing.
> Just svnsync the repository to the new server.  This will probably take a
> long time, but it does not matter since the original server can be running
> while it happens.  At the time of your choosing, do a final svnsync, and
> then shutdown the old server and use the new one.
>
> 3. Do option 1 now and then do a dump/load or svnsync at some future time
> that is more convenient for downtime.  It will probably run faster since it
> is on new and better hardware too.
>
> There are some gotchas no matter what:
>
> 1.  Does the new server have a new hostname or do you intend to update DNS
> to point to new server?  If you are not doing the latter, then all of your
> existing working copies and scripts have to be modified for the new server.
> Also any use of svn:externals property has to be modified.
>
> 2. With an old repository there is a good chance you will run into bugs in
> your data that cause svnsync or load to fail.  There are workaround for
> different failures but be prepared to run into them and account for finding
> the solutions.  SVN 1.8 and 1.9 have added various options to let you
> workaround some of the known bugs.  A common problem is having svn:
> properties that are not UTF-8 encoded or do not have LF line endings.
> svnsync has this option to work around the encoding problem:
>
> --source-prop-encoding ARG : convert translatable properties from encoding
> ARG
>  to UTF-8. If not specified, then properties are
>  presumed to be encoded in UTF-8.
>
> And it automatically fixes the LF problems.
>
> svnadmin load does not have any options to fix the problems, but you can add
> the   --bypass-prop-validation  option to ignore them and just carry the
> problems into your new repository.

Indeed, dump/load is a bit more limited than svnsync regarding
normalization. Also, be careful: both dump/load and svnsync do not
migrate your hook scripts, config files and locks (server side locks
of the svn:needs-lock type).

I've just extended the FAQ entry about dump/load with some of the
known caveats, and explained a procedure to do dump/load with minimal
downtime. See:

http://subversion.apache.org/faq.html#dumpload

-- 
Johan

Re: upgrading server

[Nico Kadel-Garcia]

On Tue, Jul 25, 2017 at 2:00 PM, Andy So  wrote:
> We have an old subversion version 1.4.3 (r23084) running on Solaris.
>
> We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9),
> possibly version 1.8.x or 1.9.x

If you're bumping Subversion release to 1.8.x or 1.9.x, I'd urge you
to go to CentOS 7, which at least starts with Subverison 1.7, and get
the latest RPM's from Wandisco.

> Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go
> through dump and load of the repository as described in various online post
> and documentations.  The repository is quite large…guessing the size to be
> in the order of 20-40GB

See above. Installing the latest Subversion on CentOS 6 gets into some
fairly awkward library dependencies, especially for the "serf"
libraries which are notably out of date on CentOS 6.

> Before we start undertaking such tasks
>
> 1.  Does anyone know if there are there any problem/gotcha in migrating
> the repository?
>
> 2.  Does anyone know how long it would take to export the repository of
> this size?  This will give us an estimate how long to schedule down time and
> cut off time.

Definitely use svnsync, if possible, so you have no downtime and can
write-lock the original repository and do a DNS based switchover.
Rollback, if there are *any* mismatched writes to the new repository,
means starting over with all working copies, which can be quite
painful.

> Thanks for any insight.

You might consider less of an extreme version update. The more
versions you update between, I've noted more likelihood of
incompatible workflows such as the changes in the "svn:external"
formats.

Re: upgrading server

[Ryan Schmidt]

Don't forget that svnadmin dump/load and svnsync don't preserve hook scripts or 
other ancillary data that might be in your repository directory on the server, 
so if you have hook scripts, authorization rules or other customizations, carry 
those over to the new server manually. And of course any configuration files 
for httpd or svnserve.

RE: upgrading server

[Andrew Reedick]

>> Does anyone know how long it would take to export the repository of this 
>> size?  This will give us an estimate how long to schedule down time and cut 
>> off time.

Svnsync is the easy option.

If you insist on doing a dump/load, then a) you can time a test run of a 
dump/load, and b) "svnadmin dump" allows you to specify revision ranges which 
means you can do incremental dumps.  You can dump/load the bulk of the repo now 
and then during the migration, run another dump/load to catch any new commits.
http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate

The only caveat I can think of is if someone changes revision properties (e.g. 
the commit message) between the time of the initial dump and the migration.  
But you can track/prevent those with a hook script.  (And is another reason why 
svnsync is preferred.)

Re: upgrading server

[David Chapman]

On 7/25/2017 11:00 AM, Andy So wrote:


We have an old subversion /version 1.4.3 (r23084) /running on Solaris.

We would like to upgrade to use new hardware on Linux based OS (CentOS 
6.9), possibly version 1.8.x or 1.9.x


Our plan is to installed and configure the latest SVN on CentOS 6.9. 
Then go through dump and load of the repository as described in 
various online post and documentations.  The repository is quite 
large…guessing the size to be in the order of 20-40GB


Before we start undertaking such tasks

1.Does anyone know if there are there any problem/gotcha in migrating 
the repository?


2.Does anyone know how long it would take to export the repository of 
this size?  This will give us an estimate how long to schedule down 
time and cut off time.


Thanks for any insight.



I recommend CentOS 7.x; CentOS 6.x is nearing the end of its support 
lifetime.  I also installed an SSD on my new machine - much faster than 
rotating media, and Subversion's "write once" philosophy (old revisions 
are essentially immutable) works well on an SSD.


--
David Chapman  dcchap...@acm.org
Chapman Consulting -- San Jose, CA
Software Development Done Right.
www.chapman-consulting-sj.com

Re: upgrading server

[Mark Phippard]

On Tue, Jul 25, 2017 at 2:00 PM, Andy So  wrote:

> We have an old subversion *version 1.4.3 (r23084) *running on Solaris.
>
> We would like to upgrade to use new hardware on Linux based OS (CentOS
> 6.9), possibly version 1.8.x or 1.9.x
>
>
>
> Our plan is to installed and configure the latest SVN on CentOS 6.9. Then
> go through dump and load of the repository as described in various online
> post and documentations.  The repository is quite large…guessing the size
> to be in the order of 20-40GB
>
>
>
> Before we start undertaking such tasks
>
> 1.  Does anyone know if there are there any problem/gotcha in
> migrating the repository?
>
> 2.  Does anyone know how long it would take to export the repository
> of this size?  This will give us an estimate how long to schedule down time
> and cut off time.
>
>
>
> Thanks for any insight.
>


Dump and load is a good idea because it lets the repository be rewritten
using the newer code and repository format which will give you a smaller
repository that will run a little faster.  That said you do not HAVE to
dump/load.  You have other options:

1. Just move the repository folder to the new server.  Perhaps using tar
and then moving the archive.

2. Instead of using dump/load, use svnsync.  This gives all the benefits of
the dump/load but allows you to shrink your downtime to almost nothing.
Just svnsync the repository to the new server.  This will probably take a
long time, but it does not matter since the original server can be running
while it happens.  At the time of your choosing, do a final svnsync, and
then shutdown the old server and use the new one.

3. Do option 1 now and then do a dump/load or svnsync at some future time
that is more convenient for downtime.  It will probably run faster since it
is on new and better hardware too.

There are some gotchas no matter what:

1.  Does the new server have a new hostname or do you intend to update DNS
to point to new server?  If you are not doing the latter, then all of your
existing working copies and scripts have to be modified for the new
server.  Also any use of svn:externals property has to be modified.

2. With an old repository there is a good chance you will run into bugs in
your data that cause svnsync or load to fail.  There are workaround for
different failures but be prepared to run into them and account for finding
the solutions.  SVN 1.8 and 1.9 have added various options to let you
workaround some of the known bugs.  A common problem is having svn:
properties that are not UTF-8 encoded or do not have LF line endings.
 svnsync has this option to work around the encoding problem:

--source-prop-encoding ARG : convert translatable properties from encoding
ARG
 to UTF-8. If not specified, then properties are
 presumed to be encoded in UTF-8.

And it automatically fixes the LF problems.

svnadmin load does not have any options to fix the problems, but you can
add the   --bypass-prop-validation  option to ignore them and just carry
the problems into your new repository.



-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

upgrading server

[Andy So]

We have an old subversion version 1.4.3 (r23084) running on Solaris.
We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9), 
possibly version 1.8.x or 1.9.x

Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go 
through dump and load of the repository as described in various online post and 
documentations.  The repository is quite large…guessing the size to be in the 
order of 20-40GB

Before we start undertaking such tasks

1.  Does anyone know if there are there any problem/gotcha in migrating the 
repository?

2.  Does anyone know how long it would take to export the repository of 
this size?  This will give us an estimate how long to schedule down time and 
cut off time.

Thanks for any insight.

Re: svn merge issues after upgrading server from 1.4.3 to 1.6.6 - unexpected property changes (deleted svn:mergeinfo)

[Gary M. Gere]

-Hi Bob,

First off, thank you very much for your quick reply.

We were running a 1.4.2 subversion server, and upgraded the subversion 
server to 1.6.6. The clients have not changed in any way, and were a mix 
of 1.4, 1.5 and 1.6 clients that used http only to communicate with 
the subversion server. We have not had any compatibility issues of any 
kind until we upgraded the server to 1.6.6.


The merge problem did not show up when the server was 1.4.2, but is now 
showing up with the 1.6.6 server.


We did not upgrade the database repository - it is still at version 3. 
I read somewhere in the 1.5 release notes that the merge tracking 
added to 1.5 will not be enabled unless the repository is upgraded, 
which it has not.


If I understand your reply correctly, what's happening is old 
svn:mergeinfo data that is no longer needed is being removed (property 
change only). Even though a large number of files are listed in the 
merge/commit, besides files that really were merged, the rest are just 
property changes.


Thank you very much.

-GmG


On 03/25/10 12:04, Bob Archer wrote:

We recently upgraded our subversion server software and are having major
problems with merging after moving the subversion server from 1.4.2 to
1.6.6. I am asking the user community for any help they can give us.

The details of the prior and current software versions are detailed at the
end of this email.

The problem we are having is with 'svn merge' with a workspace that was
checked out using the subversion 1.4.2 server:
 

Eww... not sure how great an idea it is to use a pre 1.5 server with 1.5 and 
1.6 clients. Although I'm not 100% sure of the ramifications... it could just 
be a performance issue and the client has to do more work to walk the tree of 
mergeinfo properties. Perhaps someone that knows a bit more about that issue 
will chime in here.

   

   - cddestination-branch-directory
   - svn merge -r revision1:revision2 SOURCE_SVN_URL .

This results in many unrelated files are having their properties change,
and many unrelated files being included in the merge. It makes it very
difficult to verify that the merge was successful when many unrelated
files are included.

A small example:

An attempt to merge one change from sw1.0 to sw2.0 branch.
A folder called Docs contains files that have not been changed.
An svn diff produces the following:
% cd /data/source/branches/sw2.0
% svn stat -q Docs/
  M  Docs/WDS-RIS-Blueprint.odt
  M  Docs/WDS-RIS-Script-Usage.odt

 

As you know a  M (Space in first col M in second column) indicates that the 
properties where changed but not the file itself.


   

% svn diff Docs/

Property changes on: Docs/WDS-RIS-Blueprint.odt
___
Deleted: svn:mergeinfo


Property changes on: Docs/WDS-RIS-Script-Usage.odt
___
Deleted: svn:mergeinfo
 

This shows that you do have mergeinfo on your files. So, at some time someone 
did a merge probably at the Docs level or directly to a file which added the 
mergeinfo. Now you are doing it at a higher folder level so the data is being 
elided. There's not really anything wrong here.

You  might want to train your devs to always do merges at the same folder level 
to prevent all the merge data on child folders and files. If they weren't there 
then they wouldn't be elided.


   

A mix of subversion clients have been in use, including 1.4, 1.5 and 1.6
based clients. Those have not changed, and this behavior occurs even using
a 1.6.9 client.
 

I assume it doesn't happen when you use a 1.4 client... since this version 
doesn't include the merge tracking functionality.


   

Can anyone provide any guidance, clues, pointers, etc. to what we need to
do to address this issue??
 

I think there is nothing you can do to address it. But, to solve your problem you may 
just want to pipe the output to grep (or something) and ignore files with a status of 
 M

You might also want to upgrade your server... 1.4.x is pretty old and not 
supported with updates.

I hope this helps.. sorry AFAIK there is no magic answer to make this easier.

BOb