[OpenAFS] Migrating existing data onto vice partition on the fly
Dear Users, Thank you for maintaining this list. I have been using AFS in the past, but never tried this sort of thing before. I have terabytes of data that I would like to convert into AFS. It right now exists on a large XFS filesystem. Shutting down the server, or rather, denying access to the files, even for a few hours is not really an option, not even during the holidays. I was thinking about converting this entrie large volume into an AFS space on-the-fly. Here's what I thought, would it work? Or is there a better way? First I would set up the cell and everything, then just run a vos create -server athlas -partition /vicepa -name root.afs -cell cellname -noauth ..right on top of the existing partition... (Now if this sounds extremely stupid please note I never done this before, just thinking here...) After that, I'll just locally mount my cell up under /afs/.cellname with afsclient on the same server. Set up ACL. Once that's done, since I have about enough space left to store the data 2 times over, I just get my tokens and just copy over the files to the virtual mount from the large partition, then delete the old data so that now only the AFS data exists on it. Done. Does this make sense? Or how would you do it? Thank you for any ideas. Levente
Re: [OpenAFS] Migrating existing data onto vice partition on the fly
First I would set up the cell and everything, then just run a vos create -server athlas -partition /vicepa -name root.afs -cell cellname -noauth ..right on top of the existing partition... Hmm? Describe this more. On top of what existing partition? But, ignoring that odd info above, all you have to do is: rsync -va /my-xfs/data/ /afs/yourcell/huge-empty-volume ^ |- trailing slash relevant, read rsync(1) If /my-xfs/data is writable space, you *must* to stop all writes to it (re-mount it read-only) and then run that command again to finalize things. This may or may not be downtime for you. -- Jeff Blaine kickflop.net PGP/GnuPG Key ID: 0x0C8EDD02 ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] Migrating existing data onto vice partition on the fly
Hello Jeff, Thank you for answering. I might have been a little obscure... I'll try to clear this up for you. So... Right NOW I have a partition called /vicepa, which exists right now, and has an XFS volume, which is used actively. It has some terabytes of data and about 2.5 times of free space as the actual data. This data has to be migrated within the same server to AFS. Unfortunately, I have only this one partition remotely big enough to hold the data and/or fast enough to handle in a reasonable amount of time. My idea was based on the following: Somewhere I remember reading, that it would be OK to have AFS filesystem data and normal files coexist on the same partition for a while, if I watched carefully not to run out of space. This may or may not be true - it was a long time. So my original concept was that I would create the AFS filesystem on top of the existing data on the /vicepa partition, then copy it over to the cell's logical mount, then just delete the old data and have a pure AFS partition left after. Not sure I can be more clear, but I try if you ask. Would this even work? Best Regards, Levente On 12/30/2014 04:32 PM, Jeff Blaine wrote: First I would set up the cell and everything, then just run a vos create -server athlas -partition /vicepa -name root.afs -cell cellname -noauth ..right on top of the existing partition... Hmm? Describe this more. On top of what existing partition? But, ignoring that odd info above, all you have to do is: rsync -va /my-xfs/data/ /afs/yourcell/huge-empty-volume ^ |- trailing slash relevant, read rsync(1) If /my-xfs/data is writable space, you *must* to stop all writes to it (re-mount it read-only) and then run that command again to finalize things. This may or may not be downtime for you. ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] Migrating existing data onto vice partition on the fly
Am Dienstag 30 Dezember 2014, 16:45:27 schrieb Levente Peres: Hello Jeff, Thank you for answering. I might have been a little obscure... I'll try to clear this up for you. So... Right NOW I have a partition called /vicepa, which exists right now, and has an XFS volume, which is used actively. It has some terabytes of data and about 2.5 times of free space as the actual data. Why did you mount it as /vicepa at all? It's not an AFS partition. This data has to be migrated within the same server to AFS. Why same server? An AFS setup usually consists of several servers, especially if you want to serve terabytes of data. Not to mention the needed kerberos server. Unfortunately, I have only this one partition remotely big enough to hold the data and/or fast enough to handle in a reasonable amount of time. Fast enough for what? To copy over the data or to serve it afterwards? My idea was based on the following: Somewhere I remember reading, that it would be OK to have AFS filesystem data and normal files coexist on the same partition for a while, if I watched carefully not to run out of space. This may or may not be true - it was a long time. Yes, it could work, but you should definitely test it on a spare machine first. So my original concept was that I would create the AFS filesystem on top of the existing data on the /vicepa partition, then copy it over to the cell's logical mount, then just delete the old data and have a pure AFS partition left after. You should also plan to divide the data into logical chunks that you can store on different AFS volumes (even if you stay on your single server), as it will make backups or migration of your data to other /vicepX partitions (maybe on other AFS servers) easier. HTH... Dirk -- Dirk Heinrichs dirk.heinri...@altum.de Tel: +49 (0)2471 209385 | Mobil: +49 (0)176 34473913 GPG Public Key CB614542 | Jabber: dirk.heinri...@altum.de Tox: he...@toxme.se Sichere Internetkommunikation: http://www.retroshare.org Privacy Handbuch: https://www.privacy-handbuch.de signature.asc Description: This is a digitally signed message part.
Re: [OpenAFS] Migrating existing data onto vice partition on the fly
On Tue, 2014-12-30 at 17:08 +0100, Dirk Heinrichs wrote: This data has to be migrated within the same server to AFS. Why same server? An AFS setup usually consists of several servers, especially if you want to serve terabytes of data. Not to mention the needed kerberos server. Unfortunately, I have only this one partition remotely big enough to hold the data and/or fast enough to handle in a reasonable amount of time. I think this is actually the answer to the earlier question. (and/or fast enough: adding USB externals is cheap and easy these days, but far from performant) -- brandon s allbery kf8nh sine nomine associates allber...@gmail.com ballb...@sinenomine.net unix openafs kerberos infrastructure xmonadhttp://sinenomine.net
Re: [OpenAFS] Migrating existing data onto vice partition on the fly
On 12/30/2014 05:08 PM, Dirk Heinrichs wrote: Am Dienstag 30 Dezember 2014, 16:45:27 schrieb Levente Peres: Hello Jeff, Thank you for answering. I might have been a little obscure... I'll try to clear this up for you. So... Right NOW I have a partition called /vicepa, which exists right now, and has an XFS volume, which is used actively. It has some terabytes of data and about 2.5 times of free space as the actual data. Why did you mount it as /vicepa at all? It's not an AFS partition. I originally wanted to make and AFS server... then I was crossed off, but some time ago I proved on a smaller scale my point of why I wanted AFS in the first place (cache and better protocol over WAN to name just two)... So NOW I get to do it, but... not that comfortably as a bare installation, you know... that's just how things work here... This data has to be migrated within the same server to AFS. Why same server? An AFS setup usually consists of several servers, especially if you want to serve terabytes of data. Not to mention the needed kerberos server. Same server because I need the transfer to be quick. Once I start - I need to do the migration in max 1-2 hrs... tops. I know... I know... Unfortunately, I have only this one partition remotely big enough to hold the data and/or fast enough to handle in a reasonable amount of time. Fast enough for what? To copy over the data or to serve it afterwards? Both I guess but that's not the main issue, first problem is migrating it in a very short time. So I need to do it the ugly way, within the same server... I don't have 8Gb switches or the like and storage fabric etc to assist me in this externally... simple work with what you got issue... My idea was based on the following: Somewhere I remember reading, that it would be OK to have AFS filesystem data and normal files coexist on the same partition for a while, if I watched carefully not to run out of space. This may or may not be true - it was a long time. Yes, it could work, but you should definitely test it on a spare machine first. Definitely... I will do it on a virtual environment on my home computer tonight. Hope it will work out... I guess if its OK on small number of files it should be OK with the big data too... I have backups, but... better to be sure. I wouldn't have trusted my own tests if I didn't post to this list for second opinion first, either. So my original concept was that I would create the AFS filesystem on top of the existing data on the /vicepa partition, then copy it over to the cell's logical mount, then just delete the old data and have a pure AFS partition left after. You should also plan to divide the data into logical chunks that you can store on different AFS volumes (even if you stay on your single server), as it will make backups or migration of your data to other /vicepX partitions (maybe on other AFS servers) easier. This is definitely worth taking time to think over. I have about a week to do it so I can give this some thought... thank you for pointing it out. HTH... Dirk Levente ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] Migrating existing data onto vice partition on the fly
Am Dienstag 30 Dezember 2014, 17:33:44 schrieb Levente Peres: Why same server? An AFS setup usually consists of several servers, especially if you want to serve terabytes of data. Not to mention the needed kerberos server. Same server because I need the transfer to be quick. Once I start - I need to do the migration in max 1-2 hrs... tops. I know... I know... Copy over some terabites of data in 1-2 hours? That's challenging. AFS is a network filesystem, and your data will need to go through the cache manager anyway. That means you MUST run the AFS client AND server on this same machine while it still serves the data from the old location. Still, what about the kerberos setup? Do you have one already? If not, you'll have to set it up, too (before setting up AFS). Unfortunately, I have only this one partition remotely big enough to hold the data and/or fast enough to handle in a reasonable amount of time. Fast enough for what? To copy over the data or to serve it afterwards? Both I guess but that's not the main issue, first problem is migrating it in a very short time. So I need to do it the ugly way, within the same server... I don't have 8Gb switches or the like and storage fabric etc to assist me in this externally... simple work with what you got issue... But you have rsync at your service. You can use it to copy most of the data over in the background. You can even do several iterations and then do the last iterations after ensuring that the data can't be modified anymore. And that's where you'll get into trouble. You can't mount it ro since it's also your /vicepa partition. If you serve it via NFS currently, you could re- export it ro, though. Bye... Dirk -- Dirk Heinrichs dirk.heinri...@altum.de Tel: +49 (0)2471 209385 | Mobil: +49 (0)176 34473913 GPG Public Key CB614542 | Jabber: dirk.heinri...@altum.de Tox: he...@toxme.se Sichere Internetkommunikation: http://www.retroshare.org Privacy Handbuch: https://www.privacy-handbuch.de signature.asc Description: This is a digitally signed message part.
Re: [OpenAFS] Migrating existing data onto vice partition on the fly
On 12/30/2014 11:57 AM, Dirk Heinrichs wrote: Am Dienstag 30 Dezember 2014, 17:33:44 schrieb Levente Peres: Why same server? An AFS setup usually consists of several servers, especially if you want to serve terabytes of data. Not to mention the needed kerberos server. Same server because I need the transfer to be quick. Once I start - I need to do the migration in max 1-2 hrs... tops. I know... I know... Copy over some terabites of data in 1-2 hours? That's challenging. AFS is a network filesystem, and your data will need to go through the cache manager anyway. That means you MUST run the AFS client AND server on this same machine while it still serves the data from the old location. Still, what about the kerberos setup? Do you have one already? If not, you'll have to set it up, too (before setting up AFS). Unfortunately, I have only this one partition remotely big enough to hold the data and/or fast enough to handle in a reasonable amount of time. Fast enough for what? To copy over the data or to serve it afterwards? Both I guess but that's not the main issue, first problem is migrating it in a very short time. So I need to do it the ugly way, within the same server... I don't have 8Gb switches or the like and storage fabric etc to assist me in this externally... simple work with what you got issue... But you have rsync at your service. You can use it to copy most of the data over in the background. You can even do several iterations and then do the last iterations after ensuring that the data can't be modified anymore. And that's where you'll get into trouble. You can't mount it ro since it's also your /vicepa partition. If you serve it via NFS currently, you could re- export it ro, though. Bye... Dirk You can do a bind mount where you mount the old folder as read-only (I think) in a new location, then only serve out files via AFS and the renamed read-only share. mkdir /vicepa-readonly mount /vicepa /vicepa-readonly -o ro Then share out /vicepa-readonly as a share. ** don't try this at home/work** Other crazy and dangerous options could include using something like AUFS with the RO orginal data and a new AFS layer on top, which is all mounted as a non-AFS share (security?). Tthe new data would be written to AFS, but still visible as a share until the migration is complete, then change the clients to use AFS instead of the old sharing protocol. *Here be dragons!!* Jason ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] Migrating existing data onto vice partition on the fly
On 12/30/2014 05:57 PM, Dirk Heinrichs wrote: Am Dienstag 30 Dezember 2014, 17:33:44 schrieb Levente Peres: Why same server? An AFS setup usually consists of several servers, especially if you want to serve terabytes of data. Not to mention the needed kerberos server. Same server because I need the transfer to be quick. Once I start - I need to do the migration in max 1-2 hrs... tops. I know... I know... Copy over some terabites of data in 1-2 hours? That's challenging. AFS is a network filesystem, and your data will need to go through the cache manager anyway. That means you MUST run the AFS client AND server on this same machine while it still serves the data from the old location. Yeah... painfully aware... by measurements I done last week (made a dummy AFS server and copied a few gigs over on a same RAM/CPU but with slower controller, machine to see how it went) I should be marginally OK on this side as long as I properly prepare things... Still, what about the kerberos setup? Do you have one already? If not, you'll have to set it up, too (before setting up AFS). That's the easy part... Fortunately, that's all scripted down already - how I migrate and re-create access rights, etc... along with the client-side remote setups, policies and whatnot... windows clients won't even know the difference because drive letter wont even change, they'll get/renew their tokens automatically based on AD login. Drive letters wont even change... Yes, there will be issues, there always are, but... I had ample time to fantasize about this for a while... and write scripts for my own amusement for the past year or so... been dying to do this... Unfortunately, I have only this one partition remotely big enough to hold the data and/or fast enough to handle in a reasonable amount of time. Fast enough for what? To copy over the data or to serve it afterwards? Both I guess but that's not the main issue, first problem is migrating it in a very short time. So I need to do it the ugly way, within the same server... I don't have 8Gb switches or the like and storage fabric etc to assist me in this externally... simple work with what you got issue... But you have rsync at your service. You can use it to copy most of the data over in the background. You can even do several iterations and then do the last iterations after ensuring that the data can't be modified anymore. OK that's interesting. You gave me an idea there... That would reduce the amount to be copied from terabytes to gigabytes at most... Thank you for this golden thought!! And that's where you'll get into trouble. You can't mount it ro since it's also your /vicepa partition. If you serve it via NFS currently, you could re- export it ro, though. They made me do SMB instead of NFS or AFS :-) Also, this may not necessarily be a problem... first create the AFS server, migrate data... then at the proper time, I can shut down the service completely (I have a couple hr downtime window) and do a proper differential sync. Then run the migration scripts on everything, switch over, then bring everything online... Should have no data corruption this way. Guaranteed I wont sleep for 2 days putting out fires here and there... but it's worth it... Ok... offline for now... will tell tomorrow how the home test fared... And again... how could I forget about rsync... hoooray! Bye... Dirk ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] Migrating existing data onto vice partition on the fly
Wow! More and more things to think about... Thank you Jason! I let you guys know how it goes... Levente On 12/30/2014 06:29 PM, Jason Edgecombe wrote: You can do a bind mount where you mount the old folder as read-only (I think) in a new location, then only serve out files via AFS and the renamed read-only share. mkdir /vicepa-readonly mount /vicepa /vicepa-readonly -o ro Then share out /vicepa-readonly as a share. ** don't try this at home/work** Other crazy and dangerous options could include using something like AUFS with the RO orginal data and a new AFS layer on top, which is all mounted as a non-AFS share (security?). Tthe new data would be written to AFS, but still visible as a share until the migration is complete, then change the clients to use AFS instead of the old sharing protocol. *Here be dragons!!* Jason ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] Migrating existing data onto vice partition on the fly
Levente, Same server because I need the transfer to be quick. Once I start - I need to do the migration in max 1-2 hrs... tops. I know... I know... Let's say you have exactly one terabyte of data, 1,000,000 megabytes of it. You can stretch the migration time to your indicated maximum, 2 hours, 7200 seconds. Do the math. Your disk system would have to be able to read at a constant speed of 140 megabytes/second AND write at the same 140 at the same time. And that's not taking into account the network traffic and CPU load that the AFS processes cause you, just the hardware. You can easily test whether you can meet your own requirements. # time dd if=/dev/WHATEVERITIS of=/THATPARTITION/BOGUSFILE bs=1048576 count=16384 Keep iostat running in another window to get an idea. That's the simplest it gets; absolutely sequential reads since you are reading block by block from the physical disk, and writing to one file only, on the same filesystem. I could meet your goal of 140 MB/s both ways (and exceed it by a safe margin; it gave me 237) on an old (circa 2008 I'd say?) Supermicro storage system running CentOS 6 that has an Areca ARC-1261 controller with fourteen 1TB disks in a RAID6 configuration. What's your storage system like? The question is merely posed so that you can evaluate it for yourself whether your requirements for data transfer are realistic given the hardware you have and the type of operation you intend to perform. If you have been given 2 hours to transfer 1 terabyte of data out of and back into a system that does 50 MB/s tops, by the time you need to go back online you've only completed one third of your estimated workload. So it looks like you need to think of something else... -- Atro Tossavainen, Chairman of the Board Infinite Mho Oy, Helsinki, Finland tel. +358-44-5000 600, http://www.infinitemho.fi/ ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
