Re: Need help in Subversion migration

[Bo Berglund]

On Mon, 17 May 2021 17:15:13 -0400, David Newman 
wrote:

>> A few years back I replicated our live SVN repository, to get a backup, by 
>> using
>> a dump followed by a load on the replication server. The dump files were 
>> moved
>> over the Internet in tgz files before being used to load onto the new server.
>> 
>> Then I set up svnsync to get the replica fully updated. That worked even over
>> the Internet, but my repo size was not as huge as here, only some 15 GB...
>> 
>> Now the live server has a nightly svnsync script that keeps the two repos in
>> sync. If some network issues happen so that such a sync cannot be done then 
>> the
>> following will catch up and make the replica current again.
>> 
>> Something like that but running on the local LAN (for speed) would surely be
>> possible to migrate.
>> 
>> 
>
>A faster method of copying the repository than dump while being read
>consistent would be svnadmin hotcopy.  This allows the repository to be
>open while being copied.  If possible you can NFS mount the destination
>location and hotcopy straight to it.  Then you can use incremental
>dump/restore from the source to the destination to pick up changes until
>you are ready to do the final cutover.  I've used this method many times.

I had to do the transfer while being 8400 km away...
So what I did was to create the dump files on the actual server, then zip them
and upload to my ISP webstorage using ftp. Then from there to my new server agin
FTP download and after unzipping I could load the dump files onto the new SVN
server.
That gave me a snapshot in time but all the heavy weight stuff was included.
So when I set up svnsync after a few days it figured out what should be
transfered to make the repos in sync.

Had the backup server been on the same LAN as the source server then the method
might have been different.


-- 
Bo Berglund
Developer in Sweden

Re: Need help in Subversion migration

[jackson]

awesome thorsten , it worked :)

Thanks & regards,
Jackson J

On Tue, May 18, 2021 at 7:13 PM Thorsten  wrote:

> correction
>
> *will N O T  be picked correctly
> Am 18/05/2021 um 15:40 schrieb Thorsten:
>
> Hello,
>
> Either bypass svn completly (do not use svadmin create and just rsync or
> just use tools provided by svn and do not rsync.
>
> Your mixture cannot work:
>
> svnadmin creates an empty repo: srync copys files into that repo, but
> these files belongs to your old repo, not to the new one, so they will be
> picked up correctly
> Am 18/05/2021 um 15:33 schrieb jackson:
>
> Hello Everyone ,
>
>I did a rsync , but it looks like it is not working .
>
> Below are the steps i followed for rsync
>
> *commands run on the new svn server(1.10)*
>
> svnadmin create /subversion/repos/idea
>
> chown -R apache /subversion/repos/idea
>
> rsync -a username@sourceserver:/subversion/repos/idea
> /subversion/repos/idea
>
>
> rsync worked , but when I logged into the websvn to check the revisions ,
> it was not present in the new svn console .
>
>
> *data size on both the server is same after rsync*
>
> [root@old server repos]# du -sh idea
>
> 2.8Gidea
>
>
>  [root@new server repos]# du -sh idea
>
> 2.8Gidea
>
>
>
> screenshot of idea repo in old svn server show it has 9754 rev
> [image: image.png]
>
>
>
> screenshot of idea repo in new svn server show 0 rev
> [image: image.png]
>
>
>
> Looks like i have to follow svm dump and load with incremental or svnsync
> for this migration.
>
>
> Let me try these steps and let you know.
>
>
> Thanks & Regards,
>
> Jackson J
>
> On Tue, May 18, 2021 at 8:06 AM Nathan Hartman 
> wrote:
>
>> On Mon, May 17, 2021 at 8:26 PM Daniel Shahaf 
>> wrote:
>>
>>> Mark Phippard wrote on Mon, 17 May 2021 12:24 +00:00:
>>> >
>>> > > On May 17, 2021, at 8:21 AM, jackson  wrote:
>>> > >
>>> > > 
>>> > > Hello Everyone
>>> > >
>>> > >  We are using subversion 1.7 running in LInux 6.9.
>>> > > we are working on subversion migration along with OS upgrade
>>> > >
>>> > > Currently we have built RHEL 8.3 OS with subversion 1.10 and did all
>>> the configuration , the 1.10 setup is up and working as expected .
>>> > >
>>> > > We need help in migrating the repo from current setup (v1.7 to the
>>> new setup
>>> > >
>>> > > The current setup (v1.7) has the repo size of 500GB. we are using
>>> svn load and svn dump method to migrate the repos from current setup (v1.7)
>>> to new setup(v1.10) , but it takes a lot of time due to the size of the
>>> repo (500GB) , Also the current setup (v1.7 is being currently used by the
>>> customer ,soo even after taking the dump of one particular repo and loading
>>> it in new setup , a newer revision is being made in the  current setup
>>> (v1.7 by the customer  , thus making us to do the svn dump and lsvn load
>>> again.
>>> > >
>>> > > now we need your help in identifying an efficient way in migrating
>>> the repo from the current setup (v1.7  to the ew setup(v1.10) , Need your
>>> expert advise on this please
>>> > >
>>> > >
>>> > > Also let us know we can do rsync for this?
>>> > >
>>> > > Thanks & Regards,
>>> > > Jackson J
>>> >
>>> > You do not have to do anything. Just point the new server at the
>>> > existing repositories or move them to new location.
>>>
>>> That's not necessarily correct for BDB repositories.
>>>
>>> > You can use rsync, tar or whatever method you prefer.
>>>
>>> That's not correct for live repositories.
>>>
>>> Bo and David covered the svnsync and «svnadmin hotcopy» approaches; all
>>> I have to add is that for «svnadmin dump», the --deltas, --incremental,
>>> and -r options should be used as necessary.
>>
>>
>>
>> I'll add that several subsections of "Repository Maintenance" of Chapter
>> 5 of the Subversion book are relevant, especially [1] and [2].
>>
>> [1] Migrating Repository Data Elsewhere:
>>
>> http://svnbook.red-bean.com/en/1.7/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate
>>
>> [2] Repository Replication:
>>
>> http://svnbook.red-bean.com/en/1.7/svn.reposadmin.maint.html#svn.reposadmin.maint.replication
>>
>> (In this case I think [1] is the more relevant of these.)
>>
>> Cheers,
>> Nathan
>>
>>
>>

Re: Need help in Subversion migration

[Thorsten]

correction

*will N O T  be picked correctly

Am 18/05/2021 um 15:40 schrieb Thorsten:


Hello,

Either bypass svn completly (do not use svadmin create and just rsync 
or just use tools provided by svn and do not rsync.


Your mixture cannot work:

svnadmin creates an empty repo: srync copys files into that repo, but 
these files belongs to your old repo, not to the new one, so they will 
be picked up correctly


Am 18/05/2021 um 15:33 schrieb jackson:

Hello Everyone ,

   I did a rsync , but it looks like it is not working .

Below are the steps i followed for rsync

*commands run on the new svn server(1.10)*

svnadmin create /subversion/repos/idea

chown -R apache /subversion/repos/idea

rsync -a username@sourceserver:/subversion/repos/idea 
/subversion/repos/idea



rsync worked , but when I logged into the websvn to check the 
revisions , it was not present in the new svn console .



*data size on both the server is same after rsync*

[root@old server repos]# du -sh idea

2.8G idea


[root@new server repos]# du -sh idea

2.8G idea



screenshot of idea repo in old svn server show it has 9754 rev

image.png



screenshot of idea repo in new svn server show 0 rev

image.png



Looks like i have to follow svm dump and load with incremental or 
svnsync for this migration.



Let me try these steps and let you know.


Thanks & Regards,

Jackson J


On Tue, May 18, 2021 at 8:06 AM Nathan Hartman 
mailto:hartman.nat...@gmail.com>> wrote:


On Mon, May 17, 2021 at 8:26 PM Daniel Shahaf
mailto:d...@daniel.shahaf.name>> wrote:

Mark Phippard wrote on Mon, 17 May 2021 12:24 +00:00:
>
> > On May 17, 2021, at 8:21 AM, jackson mailto:jacky3...@gmail.com>> wrote:
> >
> > 
> > Hello Everyone
> >
> >      We are using subversion 1.7 running in LInux 6.9.
> > we are working on subversion migration along with OS upgrade
> >
> > Currently we have built RHEL 8.3 OS with subversion 1.10
and did all the configuration , the 1.10 setup is up and
working as expected .
> >
> > We need help in migrating the repo from current setup
(v1.7 to the new setup
> >
> > The current setup (v1.7) has the repo size of 500GB. we
are using svn load and svn dump method to migrate the repos
from current setup (v1.7) to new setup(v1.10) , but it takes
a lot of time due to the size of the repo (500GB) , Also the
current setup (v1.7 is being currently used by the customer
,soo even after taking the dump of one particular repo and
loading it in new setup , a newer revision is being made in
the  current setup (v1.7 by the customer  , thus making us to
do the svn dump and lsvn load again.
> >
> > now we need your help in identifying an efficient way in
migrating the repo from the current setup (v1.7  to the ew
setup(v1.10) , Need your expert advise on this please
> >
> >
> > Also let us know we can do rsync for this?
> >
> > Thanks & Regards,
> > Jackson J
>
> You do not have to do anything. Just point the new server
at the
> existing repositories or move them to new location.

That's not necessarily correct for BDB repositories.

> You can use rsync, tar or whatever method you prefer.

That's not correct for live repositories.

Bo and David covered the svnsync and «svnadmin hotcopy»
approaches; all
I have to add is that for «svnadmin dump», the --deltas,
--incremental,
and -r options should be used as necessary.



I'll add that several subsections of "Repository Maintenance" of
Chapter 5 of the Subversion book are relevant, especially [1] and
[2].

[1] Migrating Repository Data Elsewhere:

http://svnbook.red-bean.com/en/1.7/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate



[2] Repository Replication:

http://svnbook.red-bean.com/en/1.7/svn.reposadmin.maint.html#svn.reposadmin.maint.replication



(In this case I think [1] is the more relevant of these.)

Cheers,
Nathan

Re: Need help in Subversion migration

[Thorsten]

Hello,

Either bypass svn completly (do not use svadmin create and just rsync or 
just use tools provided by svn and do not rsync.


Your mixture cannot work:

svnadmin creates an empty repo: srync copys files into that repo, but 
these files belongs to your old repo, not to the new one, so they will 
be picked up correctly


Am 18/05/2021 um 15:33 schrieb jackson:

Hello Everyone ,

   I did a rsync , but it looks like it is not working .

Below are the steps i followed for rsync

*commands run on the new svn server(1.10)*

svnadmin create /subversion/repos/idea

chown -R apache /subversion/repos/idea

rsync -a username@sourceserver:/subversion/repos/idea 
/subversion/repos/idea



rsync worked , but when I logged into the websvn to check the 
revisions , it was not present in the new svn console .



*data size on both the server is same after rsync*

[root@old server repos]# du -sh idea

2.8G idea


[root@new server repos]# du -sh idea

2.8G idea



screenshot of idea repo in old svn server show it has 9754 rev

image.png



screenshot of idea repo in new svn server show 0 rev

image.png



Looks like i have to follow svm dump and load with incremental or 
svnsync for this migration.



Let me try these steps and let you know.


Thanks & Regards,

Jackson J


On Tue, May 18, 2021 at 8:06 AM Nathan Hartman 
mailto:hartman.nat...@gmail.com>> wrote:


On Mon, May 17, 2021 at 8:26 PM Daniel Shahaf
mailto:d...@daniel.shahaf.name>> wrote:

Mark Phippard wrote on Mon, 17 May 2021 12:24 +00:00:
>
> > On May 17, 2021, at 8:21 AM, jackson mailto:jacky3...@gmail.com>> wrote:
> >
> > 
> > Hello Everyone
> >
> >      We are using subversion 1.7 running in LInux 6.9.
> > we are working on subversion migration along with OS upgrade
> >
> > Currently we have built RHEL 8.3 OS with subversion 1.10
and did all the configuration , the 1.10 setup is up and
working as expected .
> >
> > We need help in migrating the repo from current setup
(v1.7 to the new setup
> >
> > The current setup (v1.7) has the repo size of 500GB. we
are using svn load and svn dump method to migrate the repos
from current setup (v1.7) to new setup(v1.10) , but it takes a
lot of time due to the size of the repo (500GB) , Also the
current setup (v1.7 is being currently used by the customer
,soo even after taking the dump of one particular repo and
loading it in new setup , a newer revision is being made in
the current setup (v1.7 by the customer  , thus making us to
do the svn dump and lsvn load again.
> >
> > now we need your help in identifying an efficient way in
migrating the repo from the current setup (v1.7  to the ew
setup(v1.10) , Need your expert advise on this please
> >
> >
> > Also let us know we can do rsync for this?
> >
> > Thanks & Regards,
> > Jackson J
>
> You do not have to do anything. Just point the new server at
the
> existing repositories or move them to new location.

That's not necessarily correct for BDB repositories.

> You can use rsync, tar or whatever method you prefer.

That's not correct for live repositories.

Bo and David covered the svnsync and «svnadmin hotcopy»
approaches; all
I have to add is that for «svnadmin dump», the --deltas,
--incremental,
and -r options should be used as necessary.



I'll add that several subsections of "Repository Maintenance" of
Chapter 5 of the Subversion book are relevant, especially [1] and [2].

[1] Migrating Repository Data Elsewhere:

http://svnbook.red-bean.com/en/1.7/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate



[2] Repository Replication:

http://svnbook.red-bean.com/en/1.7/svn.reposadmin.maint.html#svn.reposadmin.maint.replication



(In this case I think [1] is the more relevant of these.)

Cheers,
Nathan

Re: Need help in Subversion migration

[jackson]

Hello Everyone ,

   I did a rsync , but it looks like it is not working .

Below are the steps i followed for rsync

*commands run on the new svn server(1.10)*

svnadmin create /subversion/repos/idea

chown -R apache /subversion/repos/idea

rsync -a username@sourceserver:/subversion/repos/idea /subversion/repos/idea


rsync worked , but when I logged into the websvn to check the revisions ,
it was not present in the new svn console .


*data size on both the server is same after rsync*

[root@old server repos]# du -sh idea

2.8Gidea


 [root@new server repos]# du -sh idea

2.8Gidea



screenshot of idea repo in old svn server show it has 9754 rev

[image: image.png]



screenshot of idea repo in new svn server show 0 rev

[image: image.png]



Looks like i have to follow svm dump and load with incremental or svnsync
for this migration.


Let me try these steps and let you know.


Thanks & Regards,

Jackson J

On Tue, May 18, 2021 at 8:06 AM Nathan Hartman 
wrote:

> On Mon, May 17, 2021 at 8:26 PM Daniel Shahaf 
> wrote:
>
>> Mark Phippard wrote on Mon, 17 May 2021 12:24 +00:00:
>> >
>> > > On May 17, 2021, at 8:21 AM, jackson  wrote:
>> > >
>> > > 
>> > > Hello Everyone
>> > >
>> > >  We are using subversion 1.7 running in LInux 6.9.
>> > > we are working on subversion migration along with OS upgrade
>> > >
>> > > Currently we have built RHEL 8.3 OS with subversion 1.10 and did all
>> the configuration , the 1.10 setup is up and working as expected .
>> > >
>> > > We need help in migrating the repo from current setup (v1.7 to the
>> new setup
>> > >
>> > > The current setup (v1.7) has the repo size of 500GB. we are using svn
>> load and svn dump method to migrate the repos from current setup (v1.7) to
>> new setup(v1.10) , but it takes a lot of time due to the size of the repo
>> (500GB) , Also the current setup (v1.7 is being currently used by the
>> customer ,soo even after taking the dump of one particular repo and loading
>> it in new setup , a newer revision is being made in the  current setup
>> (v1.7 by the customer  , thus making us to do the svn dump and lsvn load
>> again.
>> > >
>> > > now we need your help in identifying an efficient way in migrating
>> the repo from the current setup (v1.7  to the ew setup(v1.10) , Need your
>> expert advise on this please
>> > >
>> > >
>> > > Also let us know we can do rsync for this?
>> > >
>> > > Thanks & Regards,
>> > > Jackson J
>> >
>> > You do not have to do anything. Just point the new server at the
>> > existing repositories or move them to new location.
>>
>> That's not necessarily correct for BDB repositories.
>>
>> > You can use rsync, tar or whatever method you prefer.
>>
>> That's not correct for live repositories.
>>
>> Bo and David covered the svnsync and «svnadmin hotcopy» approaches; all
>> I have to add is that for «svnadmin dump», the --deltas, --incremental,
>> and -r options should be used as necessary.
>
>
>
> I'll add that several subsections of "Repository Maintenance" of Chapter 5
> of the Subversion book are relevant, especially [1] and [2].
>
> [1] Migrating Repository Data Elsewhere:
>
> http://svnbook.red-bean.com/en/1.7/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate
>
> [2] Repository Replication:
>
> http://svnbook.red-bean.com/en/1.7/svn.reposadmin.maint.html#svn.reposadmin.maint.replication
>
> (In this case I think [1] is the more relevant of these.)
>
> Cheers,
> Nathan
>
>
>