Re: [U2] UniVerse backups to disk
On 22/08/11 14:56, Robert Porter wrote: The snapshot disk space only needs to hold the amount of the changed data - not the whole filesystem. To the applications, it appears that a copy was made, but actually writes are being held behind the scenes. Don't think I'm explaining this well (Monday am), so lets try an example. Say that the UV DB is 10GB and it will take 1 hour to back it up. During 1 hour, only 1GB of data will be changed. The snapshot LV needs to be 1GB + a little for overhead - not 10GB. If you do a df while snapshot'ed it appears that 2 10GB partitions are mounted - the original and the snapshot. But the snapshot LV is actually only holding the pending writes. Unmounting the snapshot writes all the data (in order) to the real LV. While snapshot'ed the OS knows what's changed and what hasn't and UV never knows the difference. Interesting. The reason I suggested breaking the mirror was that mirroring is a common technique. As for snapshotting, there's a new linux btree-based file system on the way. It's completely copy-on-write, so effectively it's always taking snapshots until it runs out of disk and needs to delete old snapshots to make room. I don't know much about it, but I know the idea has been around for ages. There was a filesystem called TuxFS that did this, but that never made it out of beta. I think the new one owes a lot of its ideas but none of its code to TuxFS. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
Snapshotting doesn't get rid of mirroring just the need to break/merge them. I'd still suggest using mirrors. The risk of disk failure is too great. Guess you could use some other level of RAID to get there but it's hard to beat spindles plus mirrors (0+1) for databases. In fact our snapshot logical volumes are striped and mirrored as well. The snapshot volume is holding the writes, you have the same risk of a disk failure there. On the Linux front, are you talking about Btrfs (aka Better FS)?I've read some good things, but haven't gotten the nerve to play with it yet. Wols Lists antli...@youngman.org.uk 8/23/2011 6:28 AM On 22/08/11 14:56, Robert Porter wrote: ... Interesting. The reason I suggested breaking the mirror was that mirroring is a common technique. ... ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
Here is some Pie in the Sky info for ya... btrfs or Butter FS is being developed by Oracle for Linux. Its one of the reasons Oracle shutdown the development of ZFS when they bought Sun. (ZFS has many of those snapshotting features you are talking about). So now you have all of these spinoffs for ZFS with companies like: http://nexenta.com/corp/ http://www.ixsystems.com/ix/storage/titan-truenas-pro Nexenta is based on a Solaris kernel (with Debian's package management) TrueNAS is based on a BSD kernel with ZFS implemented there. ZFS never made it into the Linux kernel because of licensing issues... although there is still some attempt to do it. Apple at one point was talking about it, but, they shut the project down. So, btrfs, is the attempt to get those features you guys are talking about into the Linux kernel, and because its oracle, they will eventually do it, I'm sure (there is enough money behind it) However, as all filesystem development goes, they have been working on btrfs for some years now... and at present, it does not have a filesystem checker (fsck) that can fix errors. So its not recommend for production use. Unless of course they have just released it. I played with btrfs in its infancy a couple of years ago... at a time when you could not even boot from it. It was very finnicky, as anything new is sometimes. I use Nexenta free version for a couple of video cameras I got saddled into providing storage for. It was cheap enough to setup a spare machine I had with it. As far as using either for U2, Much testing and documenting would be required me thinks On Tue, Aug 23, 2011 at 10:30 AM, Robert Porter ropor...@ochsner.orgwrote: Snapshotting doesn't get rid of mirroring just the need to break/merge them. I'd still suggest using mirrors. The risk of disk failure is too great. Guess you could use some other level of RAID to get there but it's hard to beat spindles plus mirrors (0+1) for databases. In fact our snapshot logical volumes are striped and mirrored as well. The snapshot volume is holding the writes, you have the same risk of a disk failure there. On the Linux front, are you talking about Btrfs (aka Better FS)?I've read some good things, but haven't gotten the nerve to play with it yet. Wols Lists antli...@youngman.org.uk 8/23/2011 6:28 AM On 22/08/11 14:56, Robert Porter wrote: ... Interesting. The reason I suggested breaking the mirror was that mirroring is a common technique. ... ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- John Thompson ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
On a relevant note. I was always told that: The best way to get an accurate backup on U2 is to: -Pause the writes -Do a logical volume snapshot (*nix only) -Resume the writes -Let the logical volume snapshot finish copying off to your backup space to disk. -On top of that you should still do a regular normal backup (like uvbackup, or tar, dd, or whatever) overnight, as snapshots were never intended to be a Full Backup solution. As far as windows is concerned I can't really say, other than, my experience with Symantec (Vertias) Backup Exec was always a horrible experience. So use something else. StorageCraft, or ComVault, I've heard good things about. StorageCraft being the way cheaper option. On Tue, Aug 23, 2011 at 1:16 PM, John Thompson jthompson...@gmail.comwrote: Here is some Pie in the Sky info for ya... btrfs or Butter FS is being developed by Oracle for Linux. Its one of the reasons Oracle shutdown the development of ZFS when they bought Sun. (ZFS has many of those snapshotting features you are talking about). So now you have all of these spinoffs for ZFS with companies like: http://nexenta.com/corp/ http://www.ixsystems.com/ix/storage/titan-truenas-pro Nexenta is based on a Solaris kernel (with Debian's package management) TrueNAS is based on a BSD kernel with ZFS implemented there. ZFS never made it into the Linux kernel because of licensing issues... although there is still some attempt to do it. Apple at one point was talking about it, but, they shut the project down. So, btrfs, is the attempt to get those features you guys are talking about into the Linux kernel, and because its oracle, they will eventually do it, I'm sure (there is enough money behind it) However, as all filesystem development goes, they have been working on btrfs for some years now... and at present, it does not have a filesystem checker (fsck) that can fix errors. So its not recommend for production use. Unless of course they have just released it. I played with btrfs in its infancy a couple of years ago... at a time when you could not even boot from it. It was very finnicky, as anything new is sometimes. I use Nexenta free version for a couple of video cameras I got saddled into providing storage for. It was cheap enough to setup a spare machine I had with it. As far as using either for U2, Much testing and documenting would be required me thinks On Tue, Aug 23, 2011 at 10:30 AM, Robert Porter ropor...@ochsner.orgwrote: Snapshotting doesn't get rid of mirroring just the need to break/merge them. I'd still suggest using mirrors. The risk of disk failure is too great. Guess you could use some other level of RAID to get there but it's hard to beat spindles plus mirrors (0+1) for databases. In fact our snapshot logical volumes are striped and mirrored as well. The snapshot volume is holding the writes, you have the same risk of a disk failure there. On the Linux front, are you talking about Btrfs (aka Better FS)?I've read some good things, but haven't gotten the nerve to play with it yet. Wols Lists antli...@youngman.org.uk 8/23/2011 6:28 AM On 22/08/11 14:56, Robert Porter wrote: ... Interesting. The reason I suggested breaking the mirror was that mirroring is a common technique. ... ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- John Thompson -- John Thompson ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
take a look at http://www.pulsarsystems.com/uvback.html -- Carl Dula Voice: 973-227-8440 X111 Pulsar Systems, Inc.Fax: 973-227-8440 271 US Highway 46, STE H209 email:c...@pulsarsystems.com Fairfield, NJ 07004-2474http://www.pulsarsystems.com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
People are still splitting mirrors? From what I can see AIX 5.2 (Oct 2002!) added JFS and snapshots. Removes the risk of a mirror getting merged back the wrong way if the break-backup-merge crashes and has to be undone by hand. Breaking the mirror means either you need to have multiple mirror copies or be at risk for a disk while split. We back up 3x a day for the production database alone. Add all the other filesystems and we'd have some mirror split more often than not. That's not an acceptable risk to me. Not when there are alternatives, and better ones at that. Plus it's a journalled file system... ever had the power yanked in the middle of the day? Or a panic? The fsck's alone can take hours on a large system. With JFS they take minutes. Still have to do the UV side, but my experience has been that the files will be MUCH cleaner. Pre-JFS a unplanned downtime took roughly 24 hours - post was less than 3. Robert F. Porter, MCSE, CCNA, ZCE, OCP-Java Lead Sr. Programmer / Analyst Laboratory Information Services Ochsner Health System This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. Wols Lists antli...@youngman.org.uk 8/19/2011 6:19 PM On 18/08/11 05:07, Chris Lee wrote: ... As others have mentioned, you can pause the database. If you've got a mirror then DBPAUSE the database, break the mirror, and DBRESUME. You can now back up the broken mirror, safe in the knowledge that all the files are internally self-consistent. I think DBPAUSE guarantees the integrity of transactions, but if you backup both the /uv and the /uvaccounts directories, then you should get the transaction logs as well, I hope :-) ... ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
The snapshot disk space only needs to hold the amount of the changed data - not the whole filesystem. To the applications, it appears that a copy was made, but actually writes are being held behind the scenes. Don't think I'm explaining this well (Monday am), so lets try an example. Say that the UV DB is 10GB and it will take 1 hour to back it up. During 1 hour, only 1GB of data will be changed. The snapshot LV needs to be 1GB + a little for overhead - not 10GB. If you do a df while snapshot'ed it appears that 2 10GB partitions are mounted - the original and the snapshot. But the snapshot LV is actually only holding the pending writes. Unmounting the snapshot writes all the data (in order) to the real LV. While snapshot'ed the OS knows what's changed and what hasn't and UV never knows the difference. Robert F. Porter, MCSE, CCNA, ZCE, OCP-Java Lead Sr. Programmer / Analyst Laboratory Information Services Ochsner Health System This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. jbut...@hampshire.edu jbut...@hampshire.edu 8/20/2011 1:13 PM We don't mirror but we use filesystem snapshotting for a clean backup with less than 10 seconds of downtime: ... Only cost in this is enough disk for snapshot. Jeff Butera Sent from my iPhone ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
That's what I like... As others have mentioned, you can pause the database. If you've got a mirror then DBPAUSE the database, break the mirror, and DBRESUME. Sounds so simple yet can be so complex and expensive. There are so many ifs with the term break the mirror it's not much of a wonder that most small businesses never go down that path; as much as we'd all like to! :-) Wouldn't it be nice if we had a simple white-paper describing this functionality so the next batch of inexpensive servers we bought we could accomplish this. Presently I have to shut down the database and keep it shutdown for the 30-60 minutes the backup runs. :-(Jeeze, we have simple Windows backup scripts we've shared and would always share; you'd think U2 would have this down pat and distributed onto their site or to the wiki. Oh well, should anyone figure out a simple, effective and inexpensive way to do this sort of thing, please post it. Bill Wols Lists wrote: On 18/08/11 05:07, Chris Lee wrote: Hi All, We're running UniVerse 10.1.17 on AIX 5.3 and the backup feature included within our vendors software is pretty basic and only allows backups of Universe to tape. [snipped] [snipped] As others have mentioned, you can pause the database. If you've got a mirror then DBPAUSE the database, break the mirror, and DBRESUME. [snipped] Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
We don't mirror but we use filesystem snapshotting for a clean backup with less than 10 seconds of downtime: Dbpause Sleep few (3-5) seconds File system snapshot (5 seconds or so) Dbresume Backup from snapshot Trash snapshot Only cost in this is enough disk for snapshot. Jeff Butera Sent from my iPhone On Aug 20, 2011, at 1:51 PM, Bill Haskett wphask...@advantos.net wrote: That's what I like... As others have mentioned, you can pause the database. If you've got a mirror then DBPAUSE the database, break the mirror, and DBRESUME. Sounds so simple yet can be so complex and expensive. There are so many ifs with the term break the mirror it's not much of a wonder that most small businesses never go down that path; as much as we'd all like to! :-) Wouldn't it be nice if we had a simple white-paper describing this functionality so the next batch of inexpensive servers we bought we could accomplish this. Presently I have to shut down the database and keep it shutdown for the 30-60 minutes the backup runs. :-(Jeeze, we have simple Windows backup scripts we've shared and would always share; you'd think U2 would have this down pat and distributed onto their site or to the wiki. Oh well, should anyone figure out a simple, effective and inexpensive way to do this sort of thing, please post it. Bill Wols Lists wrote: On 18/08/11 05:07, Chris Lee wrote: Hi All, We're running UniVerse 10.1.17 on AIX 5.3 and the backup feature included within our vendors software is pretty basic and only allows backups of Universe to tape. [snipped] [snipped] As others have mentioned, you can pause the database. If you've got a mirror then DBPAUSE the database, break the mirror, and DBRESUME. [snipped] Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
Jeff: Am I mistaken to assume this is for Linux (or other variant)? See, things aren't anywhere near as easy as they seem. :-) Bill jbut...@hampshire.edu wrote: We don't mirror but we use filesystem snapshotting for a clean backup with less than 10 seconds of downtime: Dbpause Sleep few (3-5) seconds File system snapshot (5 seconds or so) Dbresume Backup from snapshot Trash snapshot Only cost in this is enough disk for snapshot. Jeff Butera Sent from my iPhone On Aug 20, 2011, at 1:51 PM, Bill Haskettwphask...@advantos.net wrote: That's what I like... As others have mentioned, you can pause the database. If you've got a mirror then DBPAUSE the database, break the mirror, and DBRESUME. Sounds so simple yet can be so complex and expensive. There are so many ifs with the term break the mirror it's not much of a wonder that most small businesses never go down that path; as much as we'd all like to! :-) Wouldn't it be nice if we had a simple white-paper describing this functionality so the next batch of inexpensive servers we bought we could accomplish this. Presently I have to shut down the database and keep it shutdown for the 30-60 minutes the backup runs. :-(Jeeze, we have simple Windows backup scripts we've shared and would always share; you'd think U2 would have this down pat and distributed onto their site or to the wiki. Oh well, should anyone figure out a simple, effective and inexpensive way to do this sort of thing, please post it. Bill Wols Lists wrote: On 18/08/11 05:07, Chris Lee wrote: Hi All, We're running UniVerse 10.1.17 on AIX 5.3 and the backup feature included within our vendors software is pretty basic and only allows backups of Universe to tape. [snipped] [snipped] As others have mentioned, you can pause the database. If you've got a mirror then DBPAUSE the database, break the mirror, and DBRESUME. [snipped] Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
Yes this is on Linux (we don't touch windows here). Same approach works on solaris and other *nix but I'm betting someone who knows windows better than I could also make it work. Jeff Butera Sent from my iPhone On Aug 20, 2011, at 6:56 PM, Bill Haskett wphask...@advantos.net wrote: Jeff: Am I mistaken to assume this is for Linux (or other variant)? See, things aren't anywhere near as easy as they seem. :-) Bill jbut...@hampshire.edu wrote: We don't mirror but we use filesystem snapshotting for a clean backup with less than 10 seconds of downtime: Dbpause Sleep few (3-5) seconds File system snapshot (5 seconds or so) Dbresume Backup from snapshot Trash snapshot Only cost in this is enough disk for snapshot. Jeff Butera Sent from my iPhone On Aug 20, 2011, at 1:51 PM, Bill Haskettwphask...@advantos.net wrote: That's what I like... As others have mentioned, you can pause the database. If you've got a mirror then DBPAUSE the database, break the mirror, and DBRESUME. Sounds so simple yet can be so complex and expensive. There are so many ifs with the term break the mirror it's not much of a wonder that most small businesses never go down that path; as much as we'd all like to! :-) Wouldn't it be nice if we had a simple white-paper describing this functionality so the next batch of inexpensive servers we bought we could accomplish this. Presently I have to shut down the database and keep it shutdown for the 30-60 minutes the backup runs. :-(Jeeze, we have simple Windows backup scripts we've shared and would always share; you'd think U2 would have this down pat and distributed onto their site or to the wiki. Oh well, should anyone figure out a simple, effective and inexpensive way to do this sort of thing, please post it. Bill Wols Lists wrote: On 18/08/11 05:07, Chris Lee wrote: Hi All, We're running UniVerse 10.1.17 on AIX 5.3 and the backup feature included within our vendors software is pretty basic and only allows backups of Universe to tape. [snipped] [snipped] As others have mentioned, you can pause the database. If you've got a mirror then DBPAUSE the database, break the mirror, and DBRESUME. [snipped] Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
On 18/08/11 05:07, Chris Lee wrote: Hi All, We're running UniVerse 10.1.17 on AIX 5.3 and the backup feature included within our vendors software is pretty basic and only allows backups of Universe to tape. I'm going to continue running backups to tape on a nightly basis, but I'd also like to automate backups over the network to another file server so that if I'm not around to swap tapes they'll continue being backed up, as well as all the benefits of having extra copies in case a tape fails. What's the best way to achieve this, is it necessary to use UniVerse's built-in uvbackup utility ? I currently have a FreeNAS server setup in another building using rsync (the rsnapshot utility - http://rsnapshot.org/) to copy the files directly at the operating system level onto the FreeNAS box, however I've no way of really verifying the validity of these files copied over as to whether the server was in the midst of rewriting a file at the time it was copied. Something nobody's mentioned - do you have raid of any sort? Could you add mirroring to your server if you don't have it already? Not knowing AIX/RS6000s, I don't know how much that would cost to add, but drives are dirt cheap nowadays. Drives certified for an IBM mini, though ... :-) The FreeNAS rsnapshot setup appears to be doing a good job at copying the files and helps me rotate the backups through hourly, daily, weekly snapshots. It tries to save disk space by hard linking the rotated snapshots and only copying the changed files, however the problem is it validates the change by timestamp on the file, and of course when uvbackup runs each night it updates the timestamp on *every* file and hence each daily snapshot I end up with another entire copy of all the files when it probably wasn't necessary. If uvbackup is the way to go, what command line options should I be looking at ? I've read the examples in IBM's Universe admin manual but it doesn't give any examples of backup to to disk paths... Our current backup is about 15GB of data to tape, backing up our entire /UVdata directory and all the accounts under that. I'd like to do the same full backups to disk. 15Gb! at a few tens of pounds/dollars for a terabyte drive, surely it's not expensive to add mirroring? From what I've pieced together in the manual the way to do this would be something like:- $ find /UVdata -print | uvbackup -f -v -l FULL UVDATA BACKUP - /remote.nfsshare/UVdata Once that's done I'm guessing I could then bundle that remote UVdata backup into a tarball and compress it to save space and keep several backups on disk. Any suggestions on the above would be greatly appreciated. Very interested to hear if uvbackup is necessary or not... I asked one of the guys from our software vendor in the past and he felt the files should be fine being backed up directly to another machine (ie: even via FTP) without uvbackup, but I'm not convinced myself that you could be 100% sure of the integrity of the files if you do a direct disk copy since how would you know if the server had some writes in a buffer it hadn't yet flushed or was in the middle of rewriting a file, or does it really not matter a great deal ? As others have mentioned, you can pause the database. If you've got a mirror then DBPAUSE the database, break the mirror, and DBRESUME. You can now back up the broken mirror, safe in the knowledge that all the files are internally self-consistent. I think DBPAUSE guarantees the integrity of transactions, but if you backup both the /uv and the /uvaccounts directories, then you should get the transaction logs as well, I hope :-) If you need to recover, you just point another instance of the UV program at your backed-up files. Only thing to bear in mind is, it must be same-endian. I've copied UV files at the disk level between SCO, NT and linux (all on Intel) and UV doesn't give a monkeys about where the file(s) came from - if the VOC pointers are okay then the file is okay. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
Chris, add the options -notag -limit 1 to your command line. The -notag stops UV from updating the header of each record after it is included in the backup. The -limit 1 stops uvbackup from trying to use the shared-memory feature, which can severely impact the performance. The uvbackup command is the only one I know of that tests the integrity of the database structures as it goes. Filesystem-based backup of UV datafiles are ok so long as all changes have been flushed to disk prior to the backup starting and no files get changes during the backup process. To be certain that the files are ok, you need to suspend database writes, do the backup, then resume database writes. Other options are to suspend database writes, tape a snapshot of the filesystem, resume database writes, mount the snapshot in another filesystem then perform the backup. Gregor -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Lee Sent: Thursday, 18 August 2011 2:07 PM To: U2 Users List Subject: [U2] UniVerse backups to disk Hi All, We're running UniVerse 10.1.17 on AIX 5.3 and the backup feature included within our vendors software is pretty basic and only allows backups of Universe to tape. I'm going to continue running backups to tape on a nightly basis, but I'd also like to automate backups over the network to another file server so that if I'm not around to swap tapes they'll continue being backed up, as well as all the benefits of having extra copies in case a tape fails. What's the best way to achieve this, is it necessary to use UniVerse's built-in uvbackup utility ? I currently have a FreeNAS server setup in another building using rsync (the rsnapshot utility - http://rsnapshot.org/) to copy the files directly at the operating system level onto the FreeNAS box, however I've no way of really verifying the validity of these files copied over as to whether the server was in the midst of rewriting a file at the time it was copied. The FreeNAS rsnapshot setup appears to be doing a good job at copying the files and helps me rotate the backups through hourly, daily, weekly snapshots. It tries to save disk space by hard linking the rotated snapshots and only copying the changed files, however the problem is it validates the change by timestamp on the file, and of course when uvbackup runs each night it updates the timestamp on *every* file and hence each daily snapshot I end up with another entire copy of all the files when it probably wasn't necessary. If uvbackup is the way to go, what command line options should I be looking at ? I've read the examples in IBM's Universe admin manual but it doesn't give any examples of backup to to disk paths... Our current backup is about 15GB of data to tape, backing up our entire /UVdata directory and all the accounts under that. I'd like to do the same full backups to disk. From what I've pieced together in the manual the way to do this would be something like:- $ find /UVdata -print | uvbackup -f -v -l FULL UVDATA BACKUP - /remote.nfsshare/UVdata Once that's done I'm guessing I could then bundle that remote UVdata backup into a tarball and compress it to save space and keep several backups on disk. Any suggestions on the above would be greatly appreciated. Very interested to hear if uvbackup is necessary or not... I asked one of the guys from our software vendor in the past and he felt the files should be fine being backed up directly to another machine (ie: even via FTP) without uvbackup, but I'm not convinced myself that you could be 100% sure of the integrity of the files if you do a direct disk copy since how would you know if the server had some writes in a buffer it hadn't yet flushed or was in the middle of rewriting a file, or does it really not matter a great deal ? Thanks, Chris ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- Message protected by DealerGuard: e-mail anti-virus, anti-spam and content filtering. http://www.pentanasolutions.com Click here to report this message as spam: https://login.mailguard.com.au/report/1CWgZEQvxW/1HpKcaS1Mr3xxV6sO44hVn/0 This email and any attachments to it are confidential. You must not use, disclose or act on the email if you are not the intended recipient. Liability limited by a scheme approved under Professional Standards Legislation. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
Chris Lee-17 wrote: What's the best way to achieve this, is it necessary to use UniVerse's built-in uvbackup utility ? In the past for a small client who had a overnight 'dead' period, I used a simple windows script (UniVerse on MS-Windows) to backup using uvbackup to a scratch disk, which was flagged to be backed up later, via the network backup server. The backup software did not have a plugin for UniVerse (I'm not sure that any enterprise backup software does) as is common for MS-Exchange/MS-SQL etc. which are data structure and transaction processing aware. The issue with backing up out of the UniVerse environment is the lack of control over incomplete transactions and caching. Always best to use uvbackup if possible. - Learn and Do Excel and Share http://mvdbs.com http://mvdbs.com -- View this message in context: http://old.nabble.com/UniVerse-backups-to-disk-tp32284751p32289120.html Sent from the U2 - Users mailing list archive at Nabble.com. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] UniVerse backups to disk
I do an additional archival backup with cp at the linux OS level before and after our month-end batch processing since that's the data we typically have to refer back to. I bought a 4TB iomega NAS appliance that was relatively inexpensive just for this purpose. The archive share is NFS mounted on the UV server. We're fortunate enough to have times outside normal business hours when UV is relatively quiescent so I don't have to pause the database. The cp command gets executed within a UV program at the beginning and end of the batch process: PRINT '[SU0011] Creating archive directory' EXECUTE 'SH -c mkdir ':MOUNT.POINT:'/':DIR.NAME:'' PRINT '[SU0011] Archiving FABRIC.PROD at ':OCONV(TIME(),'MT') EXECUTE 'SH -c cp -r ':@PATH:' ':MOUNT.POINT:'/':DIR.NAME:'' PRINT '[SU0011] Archival completed at ':OCONV(TIME(),'MT') PRINT '[SU0011] Archival completed at ':OCONV(TIME(),'MT') PRINT '[SU0011] Removing write permissions' EXECUTE 'SH -c chmod -R ugo-w ':MOUNT.POINT:'/':DIR.NAME:'' PRINT '[SU0011] All actions completed at ':OCONV(TIME(),'MT') The DIR.NAME variable contains the current month and year, so it was very easy to write a utility that non-admin UV programmers can use to access archive data on the NFS share just by supplying a month, year, and file name. -John -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Chris Lee Sent: Wednesday, August 17, 2011 9:07 PM To: U2 Users List Subject: [U2] UniVerse backups to disk Hi All, We're running UniVerse 10.1.17 on AIX 5.3 and the backup feature included within our vendors software is pretty basic and only allows backups of Universe to tape. I'm going to continue running backups to tape on a nightly basis, but I'd also like to automate backups over the network to another file server so that if I'm not around to swap tapes they'll continue being backed up, as well as all the benefits of having extra copies in case a tape fails. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
[U2] UniVerse backups to disk
Hi All, We're running UniVerse 10.1.17 on AIX 5.3 and the backup feature included within our vendors software is pretty basic and only allows backups of Universe to tape. I'm going to continue running backups to tape on a nightly basis, but I'd also like to automate backups over the network to another file server so that if I'm not around to swap tapes they'll continue being backed up, as well as all the benefits of having extra copies in case a tape fails. What's the best way to achieve this, is it necessary to use UniVerse's built-in uvbackup utility ? I currently have a FreeNAS server setup in another building using rsync (the rsnapshot utility - http://rsnapshot.org/) to copy the files directly at the operating system level onto the FreeNAS box, however I've no way of really verifying the validity of these files copied over as to whether the server was in the midst of rewriting a file at the time it was copied. The FreeNAS rsnapshot setup appears to be doing a good job at copying the files and helps me rotate the backups through hourly, daily, weekly snapshots. It tries to save disk space by hard linking the rotated snapshots and only copying the changed files, however the problem is it validates the change by timestamp on the file, and of course when uvbackup runs each night it updates the timestamp on *every* file and hence each daily snapshot I end up with another entire copy of all the files when it probably wasn't necessary. If uvbackup is the way to go, what command line options should I be looking at ? I've read the examples in IBM's Universe admin manual but it doesn't give any examples of backup to to disk paths... Our current backup is about 15GB of data to tape, backing up our entire /UVdata directory and all the accounts under that. I'd like to do the same full backups to disk. From what I've pieced together in the manual the way to do this would be something like:- $ find /UVdata -print | uvbackup -f -v -l FULL UVDATA BACKUP - /remote.nfsshare/UVdata Once that's done I'm guessing I could then bundle that remote UVdata backup into a tarball and compress it to save space and keep several backups on disk. Any suggestions on the above would be greatly appreciated. Very interested to hear if uvbackup is necessary or not... I asked one of the guys from our software vendor in the past and he felt the files should be fine being backed up directly to another machine (ie: even via FTP) without uvbackup, but I'm not convinced myself that you could be 100% sure of the integrity of the files if you do a direct disk copy since how would you know if the server had some writes in a buffer it hadn't yet flushed or was in the middle of rewriting a file, or does it really not matter a great deal ? Thanks, Chris ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
